-- Logs begin at Tue 2022-07-26 14:32:43 UTC, end at Tue 2022-07-26 15:54:15 UTC. -- Jul 26 15:14:48.639700 dv-cinder-851032-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 15:14:48.649141 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 15:14:48 2022] *** Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: nodename: dv-cinder-851032-1-iscsi Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: machine: x86_64 Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: clock source: unix Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: pcre jit disabled Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: detected number of CPU cores: 4 Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: current working directory: / Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: detected binary path: /usr/bin/uwsgi-core Jul 26 15:14:48.651191 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: your processes number limit is 47680 Jul 26 15:14:48.651911 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: your memory page size is 4096 bytes Jul 26 15:14:48.651911 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: detected max file descriptor number: 2048 Jul 26 15:14:48.651911 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: lock engine: pthread robust mutexes Jul 26 15:14:48.651911 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: thunder lock: enabled Jul 26 15:14:48.651911 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 15:14:48.651911 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 15:14:48.690626 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: Python main interpreter initialized at 0x560c10d399c0 Jul 26 15:14:48.690626 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: python threads support enabled Jul 26 15:14:48.690626 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: your server socket listen backlog is limited to 100 connections Jul 26 15:14:48.690626 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: your mercy for graceful operations on workers is 90 seconds Jul 26 15:14:48.696199 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 15:14:48.696199 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: *** Operational MODE: preforking *** Jul 26 15:14:48.696199 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: *** uWSGI is running in multiple interpreter mode *** Jul 26 15:14:48.696199 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: spawned uWSGI master process (pid: 80364) Jul 26 15:14:48.696199 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: spawned uWSGI worker 1 (pid: 80377, cores: 1) Jul 26 15:14:48.696199 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: spawned uWSGI worker 2 (pid: 80378, cores: 1) Jul 26 15:14:48.696199 dv-cinder-851032-1-iscsi devstack@keystone.service[80364]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 15:14:48.691524 dv-cinder-851032-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 15:14:54.278810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 15:14:54.278810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: from cryptography.utils import int_from_bytes Jul 26 15:14:54.279608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 15:14:54.279608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: from cryptography.utils import int_from_bytes Jul 26 15:14:54.285620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 15:14:54.285620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Jul 26 15:14:54.286072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 15:14:54.286072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Jul 26 15:14:54.329522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 15:14:54.329710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 15:14:54.329894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] command line args: [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 15:14:54.330167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 15:14:54.330348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ================================================================================ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 15:14:54.330603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] admin_token = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.330848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config_dir = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.331037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.331222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config_source = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.331403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] debug = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.331603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_config_append = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_dir = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.333787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.334022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.334268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.334387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 15:14:54.334515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.334601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 15:14:54.334725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.334812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] command line args: [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 15:14:54.334890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.335009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 15:14:54.335090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.335188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ================================================================================ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 15:14:54.335875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.336118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.336280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.336410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.336727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_dir = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.336871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.336955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_source = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] debug = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.337796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.128/identity {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] publish_errors = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_config_append = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.338961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_dir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] standard_threads = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.339851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_journal = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_json = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_stderr = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.340932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_syslog = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.341067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.341151 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.341301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.341383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.341533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.341620 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.341765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.341849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.341995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.342092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.342240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.342342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.342483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.342582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.342755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.342834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.343056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.343256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.128/identity {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.343332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.343440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] publish_errors = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.343592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.343670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.343776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.343891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.344019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.344101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.344268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.344346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.344526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.344607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.344771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] standard_threads = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.344861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.345000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.345079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.345241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.345317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.345415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.345491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.345591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_journal = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.345738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.345816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_json = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.345927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_stderr = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.346008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.346091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_syslog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.346209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.346317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 15:14:54.346463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.346571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.346724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.346808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.346977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347774 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.347923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.348916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.349934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.350894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.351640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.351970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.352066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.352206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.352397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.352522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.352689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.352779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.352942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.353880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.354892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.355974 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.356946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.357945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.358895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.359984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.360957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.361962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.362923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.363851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.364958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.365976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.366139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.366215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.366376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.366462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.366682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.366765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.366919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.external = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.367139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.367297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.367403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.367602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.367680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.367840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.367919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.password = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.token = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368814 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.368904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.369884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.370996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.371074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.371232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.371318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.371476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.371552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.371712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.371788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.372007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.372404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.372625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.372704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.372862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.external = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.373197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.373357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.373450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.373654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.373734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.373888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.373965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.password = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.token = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.374993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.375976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.376967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.377942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378261 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.378986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.379955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.380122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.380230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.380366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.380540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.380667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.380805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.380932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.381959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.382095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.382761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.382966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.383091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.383331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.383461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.383704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.383825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.383959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.384969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.385144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.385296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.385420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.385420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.385691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.385843 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.385967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.386991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.387122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.387245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.387374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.387498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.387620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.387743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.387877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.388945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.389954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.390983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.391109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.391234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.391359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.391486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.391619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.391777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.391906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.392939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.393949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.394952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.395956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.396996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.397123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.397250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.397376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.397502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.397636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.397884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.398552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.398680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.398804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.398928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.399969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.400985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.401989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.402111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.402304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.402431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.402554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.402676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.402809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.402932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.403924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.404988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.405120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.405255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.405381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.405508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.405635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.405760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.405886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.406935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.407925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.408835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.409091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.409219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.409349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.409475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.409602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.409728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.409909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.410915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.411917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.412981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.413107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.413241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.413367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.413548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.413737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.413890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.414875 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.415860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.416107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.416232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.416374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.416545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.416545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.416803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.416886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.417992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.418240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.418366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.418501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.418639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.418774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.418901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419668 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.419927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.420973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.421100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.421226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.421353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.421491 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.421619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.421744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.421870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422774 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.422899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.423399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.424548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.425589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.425589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.425589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 15:14:54.425589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 15:14:54.450455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.450713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.450908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.451862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.452094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.452318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.459522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.462185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.490229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.490483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.491829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.492225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.544533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.545855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.596064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.596339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.597146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.597386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.662162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.662415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.662582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.662722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.662854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.662985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.663114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.663240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.663900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.664032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.664159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.664283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.687503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.687724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.687955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.688184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.688405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.688649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.688816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.688959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.689084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.689269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.689528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.689755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.689975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.690190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.700787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.701434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.713736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.713959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.714187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.714322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.714454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.714589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.714727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.714858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.714987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.715116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.715245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.715382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.715382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.715636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 15:14:54.732724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.732936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 15:14:54.733276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.733444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.733577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 15:14:54.733703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 15:14:54.733830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.733983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.734120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.734257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 15:14:54.734383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.734507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.734631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.734755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.734943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.735201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.744584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.744866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.745138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.745355 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.745490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.745633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.745892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.746176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.752670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.753827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.758102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.758382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.758670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.758955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.759238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.759496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.759625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.759750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.759873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.760031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.760156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.760361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.760520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.760665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.760752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.760938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.761123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.761248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.761462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.761733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.762003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.762269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.784109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.785627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.785913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.786053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.787451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.787738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.790599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.790877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.791162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.792763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.793045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.793327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.797961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.798210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.799952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.800235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.802401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.802692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.802984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.803267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.803543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.803829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.804753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.804943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.805287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.805580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.805864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.806142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.806414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.806697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.807608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.807608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.821191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.821482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.823923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.824199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.825458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.828129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.828631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.828933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.831296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.831594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.833371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.836842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.837168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.840376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.840704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.840797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.840996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.841294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.841592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.841890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.843909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.844209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.844512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.844797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.845072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.845365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.858161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.858448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.858714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.858977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.861902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.862188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.862456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.862719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.865906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.868001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.868278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.870831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.871132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.871419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.872164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.872164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.872393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.872500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.872761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.875702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.875979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.876268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.876583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.876870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.884198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.884500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.884788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.885071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.885352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.885625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.885922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.888704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.888961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.889243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.889530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.889805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.890077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.890351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.902214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.905211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.907157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.908171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.910267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.910702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.912236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.912546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.912834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.914752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.916283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.917925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.918228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.918511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.919464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.921088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.924037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.924331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.924640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.924929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.925162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.926645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.930408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.930696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.930994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.931282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.933651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.933940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.934225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.939491 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.939778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.940068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.941150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.941431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.941714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.941991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.942278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.942561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.942835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.943119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.943499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.943783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.944071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.944336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.944696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.945463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.947936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 15:14:54.948209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.948513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.948918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.949200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.949470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.949742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.950013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.950288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.950556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.950846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.951120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.951384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.951642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 15:14:54.979233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560c10d399c0 pid: 80377 (default app) Jul 26 15:14:54.982640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288af522-e0d0-4429-ab7d-34371bca7973 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:54.982928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288af522-e0d0-4429-ab7d-34371bca7973 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:54.983210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288af522-e0d0-4429-ab7d-34371bca7973 None None] PATH_INFO: `/v3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:54.986863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560c10d399c0 pid: 80378 (default app) Jul 26 15:14:54.987519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1/1] 10.222.0.128 () {50 vars in 796 bytes} [Tue Jul 26 15:14:54 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:03.495035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e59d1-ab07-4d6d-983a-121c14a98d97 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:03.495594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e59d1-ab07-4d6d-983a-121c14a98d97 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:03.495594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e59d1-ab07-4d6d-983a-121c14a98d97 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:03.500105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1/2] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:03 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:03.505848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:03.506456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:03.506587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:03.579407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: INFO dbcounter [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] Registered counter for database keystone Jul 26 15:15:03.601133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG oslo_db.sqlalchemy.engines [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80377) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 15:15:03.607635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writer thread running {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 15:15:03.787857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG passlib.handlers.bcrypt [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80377) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 15:15:03.788362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG passlib.handlers.bcrypt [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80377) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 15:15:03.844758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:03.846416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:03.981533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 2/3] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:03 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 15:15:04.121632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e59d1-ab07-4d6d-983a-121c14a98d97 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:04.191177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: INFO dbcounter [None req-a77e59d1-ab07-4d6d-983a-121c14a98d97 None None] Registered counter for database keystone Jul 26 15:15:04.210977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG oslo_db.sqlalchemy.engines [None req-a77e59d1-ab07-4d6d-983a-121c14a98d97 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80378) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 15:15:04.217296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writer thread running {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 15:15:04.341194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e59d1-ab07-4d6d-983a-121c14a98d97 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:04.345465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5c650-4d02-405e-bf00-e0689c2d8a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:04.346102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5c650-4d02-405e-bf00-e0689c2d8a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:04.346389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5c650-4d02-405e-bf00-e0689c2d8a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:04.346668 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5c650-4d02-405e-bf00-e0689c2d8a28 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:04.355506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5c650-4d02-405e-bf00-e0689c2d8a28 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:04.407793 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING keystone.common.rbac_enforcer.enforcer [None req-30f5c650-4d02-405e-bf00-e0689c2d8a28 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 15:15:04.408064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5c650-4d02-405e-bf00-e0689c2d8a28 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:04.419072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 2/4] 10.222.0.128 () {58 vars in 1329 bytes} [Tue Jul 26 15:15:04 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:15:04.425061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:04.439785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439e907-3744-4bd3-87ee-d886cc0ef677 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:04.444328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3d8d8-9def-4db5-bcd3-cf10a30c519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:04.445534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3d8d8-9def-4db5-bcd3-cf10a30c519e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:04.445815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3d8d8-9def-4db5-bcd3-cf10a30c519e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:04.446088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3d8d8-9def-4db5-bcd3-cf10a30c519e admin admin] PATH_INFO: `/v3/projects/b15b339f34b0459888ba2e587e895d4f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:04.447686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3d8d8-9def-4db5-bcd3-cf10a30c519e admin admin] RBAC: Authorizing `identity:get_project(project_id=b15b339f34b0459888ba2e587e895d4f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:04.498556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2ff3d8d8-9def-4db5-bcd3-cf10a30c519e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 15:15:04.498903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3d8d8-9def-4db5-bcd3-cf10a30c519e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:04.503592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 3/5] 10.222.0.128 () {56 vars in 1212 bytes} [Tue Jul 26 15:15:04 2022] GET /identity/v3/projects/b15b339f34b0459888ba2e587e895d4f => generated 345 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:06.513944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd19887-54f8-48af-9fb9-607a2a9a106e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:06.514434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd19887-54f8-48af-9fb9-607a2a9a106e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:06.514592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd19887-54f8-48af-9fb9-607a2a9a106e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:06.518610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 3/6] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:06 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:06.524372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452f27b-4230-4fec-93a8-5f3ed2e08301 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:06.524834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452f27b-4230-4fec-93a8-5f3ed2e08301 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:06.524977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452f27b-4230-4fec-93a8-5f3ed2e08301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:06.545046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4452f27b-4230-4fec-93a8-5f3ed2e08301 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:06.546555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4452f27b-4230-4fec-93a8-5f3ed2e08301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:06.554939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 4/7] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:06 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 15:15:06.584678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd19887-54f8-48af-9fb9-607a2a9a106e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:06.597090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd19887-54f8-48af-9fb9-607a2a9a106e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:06.601868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bc5a4-47da-49a7-89cf-79c2033adc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:06.602478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc5a4-47da-49a7-89cf-79c2033adc3c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:06.602765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc5a4-47da-49a7-89cf-79c2033adc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:06.603043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc5a4-47da-49a7-89cf-79c2033adc3c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:06.610678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc5a4-47da-49a7-89cf-79c2033adc3c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:06.612448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc5a4-47da-49a7-89cf-79c2033adc3c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:06.622496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 4/8] 10.222.0.128 () {58 vars in 1329 bytes} [Tue Jul 26 15:15:06 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:15:06.628088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452f27b-4230-4fec-93a8-5f3ed2e08301 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:06.637177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452f27b-4230-4fec-93a8-5f3ed2e08301 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:06.641652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c952e-80b2-46ed-90dd-b3305e974a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:06.642284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c952e-80b2-46ed-90dd-b3305e974a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:06.642567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c952e-80b2-46ed-90dd-b3305e974a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:06.642842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c952e-80b2-46ed-90dd-b3305e974a97 admin admin] PATH_INFO: `/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:06.644210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c952e-80b2-46ed-90dd-b3305e974a97 admin admin] RBAC: Authorizing `identity:get_user(user_id=7e2685b786dd4d2eb4cf48fedfc3b189)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:06.646079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c952e-80b2-46ed-90dd-b3305e974a97 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:06.647646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 5/9] 10.222.0.128 () {56 vars in 1203 bytes} [Tue Jul 26 15:15:06 2022] GET /identity/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189 => generated 248 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:08.747314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8484f-4117-45be-869c-63a5adbb8c0a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:08.747831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8484f-4117-45be-869c-63a5adbb8c0a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:08.747922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8484f-4117-45be-869c-63a5adbb8c0a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:08.749268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 5/10] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:08.754482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7241f-1da9-446d-8bda-f192211dba30 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:08.754910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7241f-1da9-446d-8bda-f192211dba30 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:08.755046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7241f-1da9-446d-8bda-f192211dba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:08.775351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d2f7241f-1da9-446d-8bda-f192211dba30 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:08.776646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d2f7241f-1da9-446d-8bda-f192211dba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:08.796096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67271-a52f-4389-aaad-d5c843879736 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:08.796391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67271-a52f-4389-aaad-d5c843879736 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:08.796702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67271-a52f-4389-aaad-d5c843879736 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:08.798034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 6/11] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:08.802974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:08.803298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:08.803575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:08.845971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80378) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 15:15:08.846407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80378) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 15:15:08.865962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 6/12] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:08.892330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:08.892654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7241f-1da9-446d-8bda-f192211dba30 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:08.893976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:08.920482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:08.996384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 7/13] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:09.005223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7241f-1da9-446d-8bda-f192211dba30 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.019140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:09.041104 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.041763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.042035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.042313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.054016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:09.063053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:09.064006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:09.064981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 7/14] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:09.070793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:09.124822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd354b-45b1-4131-9e39-8d249a9c742e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.127544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:09.152577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:09.165008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.165895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.166187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.166473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.170950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bdc8-19ff-4e57-9b18-07ecace45d7c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.209161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.210595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.210955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.211235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.211977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:09.212286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:09.213928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:09.226609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 8/15] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:09 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:09.232227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:09.239589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:09.241835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:09.337831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedbaf7-7168-4ac2-ab88-2e43fa87becd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.342598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 8/16] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:15:09.348717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:09.352710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:09.459933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2aeb3a-c720-46de-a594-a2d49ae11cfa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.481083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.481745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.482033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.482313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] PATH_INFO: `/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.483495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] RBAC: Authorizing `identity:get_user(user_id=7e2685b786dd4d2eb4cf48fedfc3b189)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:09.484997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:09.486599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 9/17] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:09 2022] GET /identity/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189 => generated 248 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:09.492004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:09.499454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.500337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.501059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.501336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.507422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:09.524128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:09.525924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:15:09.544891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:09.545778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:09.546604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:15:09.546604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:15:09.546604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:15:09.546604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:09.546604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 9/18] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:09.551881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:09.596820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526d98e-65e5-4309-98bc-2d647903cf45 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.609279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:09.632098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.632776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.633062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.633350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.634518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:09.635971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:09.637590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 10/19] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:09 2022] GET /identity/v3/domains/default => generated 200 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:09.643254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:09.655124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12278b38-7b2b-49d3-8797-96ca96cf1b07 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.689012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e473f1-b5c7-496b-8851-db0a14c3588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.689642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e473f1-b5c7-496b-8851-db0a14c3588b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.689932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e473f1-b5c7-496b-8851-db0a14c3588b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.690212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e473f1-b5c7-496b-8851-db0a14c3588b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.690923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e473f1-b5c7-496b-8851-db0a14c3588b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:09.691234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e473f1-b5c7-496b-8851-db0a14c3588b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:09.692663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e473f1-b5c7-496b-8851-db0a14c3588b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:09.711402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 10/20] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:15:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:09.724379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:09.746702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785d89-c7fe-4d54-b277-e37a9a1f2951 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:09.789113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1f6b2-6d6c-4e4c-a4ec-15ff47eab621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:09.789800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1f6b2-6d6c-4e4c-a4ec-15ff47eab621 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:09.790025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1f6b2-6d6c-4e4c-a4ec-15ff47eab621 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:09.791456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1f6b2-6d6c-4e4c-a4ec-15ff47eab621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:09.791456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1f6b2-6d6c-4e4c-a4ec-15ff47eab621 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.domain.id=default, user.id=7e2685b786dd4d2eb4cf48fedfc3b189) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:09.791655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1f6b2-6d6c-4e4c-a4ec-15ff47eab621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:09.793138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1f6b2-6d6c-4e4c-a4ec-15ff47eab621 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:09.800737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 11/21] 10.222.0.128 () {56 vars in 1327 bytes} [Tue Jul 26 15:15:09 2022] GET /identity/v3/role_assignments?user.id=7e2685b786dd4d2eb4cf48fedfc3b189&scope.domain.id=default&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 234 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:12.085264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33fd0c-9b98-48d5-ad89-f2a050f62330 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.086324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33fd0c-9b98-48d5-ad89-f2a050f62330 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.086817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33fd0c-9b98-48d5-ad89-f2a050f62330 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.088515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 11/22] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:12 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:12.095713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ccaa2-08b6-4f56-a6e7-353d0ded139f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.096617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ccaa2-08b6-4f56-a6e7-353d0ded139f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.097127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ccaa2-08b6-4f56-a6e7-353d0ded139f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.125484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-740ccaa2-08b6-4f56-a6e7-353d0ded139f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:12.127049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-740ccaa2-08b6-4f56-a6e7-353d0ded139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:12.185009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3079f-4395-4970-a5ef-e79b0d6ae3e0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.317254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3079f-4395-4970-a5ef-e79b0d6ae3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.317254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3079f-4395-4970-a5ef-e79b0d6ae3e0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.317254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 12/23] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:12.317254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c2167-e830-4bd8-9397-c0a139ae0c8b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.317254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c2167-e830-4bd8-9397-c0a139ae0c8b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.317254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c2167-e830-4bd8-9397-c0a139ae0c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.318337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-763c2167-e830-4bd8-9397-c0a139ae0c8b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:12.318527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 12/24] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:12.318683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-763c2167-e830-4bd8-9397-c0a139ae0c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:12.318832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ccaa2-08b6-4f56-a6e7-353d0ded139f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:12.402651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 13/25] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:12.443103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:12.471858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c2167-e830-4bd8-9397-c0a139ae0c8b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:12.487014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ccaa2-08b6-4f56-a6e7-353d0ded139f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:12.536775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5e356-0910-4f94-a54c-2a4e8bbcb6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:12.538208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5e356-0910-4f94-a54c-2a4e8bbcb6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.538601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5e356-0910-4f94-a54c-2a4e8bbcb6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.538989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5e356-0910-4f94-a54c-2a4e8bbcb6f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.565747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5e356-0910-4f94-a54c-2a4e8bbcb6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:12.567296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5e356-0910-4f94-a54c-2a4e8bbcb6f5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:12.587647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-adb5e356-0910-4f94-a54c-2a4e8bbcb6f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:15:12.596336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:12.597425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:12.598441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:15:12.598441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:15:12.598441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:15:12.598441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:12.601306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 13/26] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:12.603152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d032ae-8b5d-4ed5-9a16-f27852f20085 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.603636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d032ae-8b5d-4ed5-9a16-f27852f20085 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.604094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d032ae-8b5d-4ed5-9a16-f27852f20085 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.605633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 14/27] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:12.615208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28ee1e-06fb-43d6-bbfd-04a7b65ed6b2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.615208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28ee1e-06fb-43d6-bbfd-04a7b65ed6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.615208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28ee1e-06fb-43d6-bbfd-04a7b65ed6b2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.615208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 15/28] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:12.615208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28ee1e-06fb-43d6-bbfd-04a7b65ed6b2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:12.667715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c2167-e830-4bd8-9397-c0a139ae0c8b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:12.713213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dad651-112f-4dd0-852f-401ac78e4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:12.714270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dad651-112f-4dd0-852f-401ac78e4385 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.714822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dad651-112f-4dd0-852f-401ac78e4385 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.715321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dad651-112f-4dd0-852f-401ac78e4385 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.731823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dad651-112f-4dd0-852f-401ac78e4385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-73dad651-112f-4dd0-852f-401ac78e4385 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:12.741334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:12.742307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:12.743066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:12.745669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 14/29] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:12.747612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596079-9418-46bc-af92-1c23a2971057 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.748269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596079-9418-46bc-af92-1c23a2971057 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.748854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596079-9418-46bc-af92-1c23a2971057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.768119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28ee1e-06fb-43d6-bbfd-04a7b65ed6b2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:12.771377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-03596079-9418-46bc-af92-1c23a2971057 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:12.773687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-03596079-9418-46bc-af92-1c23a2971057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:12.804057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:12.816867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45408771-f255-4518-9d4b-bcd5bd788ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:12.823954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45408771-f255-4518-9d4b-bcd5bd788ebd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.823954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45408771-f255-4518-9d4b-bcd5bd788ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.823954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45408771-f255-4518-9d4b-bcd5bd788ebd None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.823954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45408771-f255-4518-9d4b-bcd5bd788ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:12.823954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45408771-f255-4518-9d4b-bcd5bd788ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:12.823954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45408771-f255-4518-9d4b-bcd5bd788ebd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:12.844816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 16/30] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:15:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:12.852618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684afca0-1be7-42f0-8821-0955582ac954 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:12.852618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684afca0-1be7-42f0-8821-0955582ac954 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:12.852618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684afca0-1be7-42f0-8821-0955582ac954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:12.872740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 15/31] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:12.872740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03596079-9418-46bc-af92-1c23a2971057 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:12.889336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-684afca0-1be7-42f0-8821-0955582ac954 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:12.892410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-684afca0-1be7-42f0-8821-0955582ac954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:12.919770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:12.985313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 17/32] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:12.986706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684afca0-1be7-42f0-8821-0955582ac954 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:13.107629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03596079-9418-46bc-af92-1c23a2971057 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:13.125273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684afca0-1be7-42f0-8821-0955582ac954 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:13.146508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:13.147149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.147620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.147904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.148727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:13.148977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:13.150605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:13.151828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:13.165323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 16/33] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:12 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:13.168258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:13.176758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:13.178717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.179796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.180295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.181538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:13.184432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:13.271762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:13.275958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fbea-51ea-4fc8-87e3-c69d31632f89 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:13.312013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:13.312686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.312980 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.313260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.314037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:13.315435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:13.446813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 18/34] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:15:12 2022] POST /identity/v3/projects => generated 302 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:13.448358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:13.525472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 17/35] 10.222.0.128 () {60 vars in 1136 bytes} [Tue Jul 26 15:15:13 2022] POST /identity/v3/roles => generated 220 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:13.526520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:13.673460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e634a0-f825-46b4-960d-f5b071f0c731 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:13.700763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d599598-8983-4e63-8769-fcd815cd03ca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:13.725134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0263d-3f44-432b-8528-6fc4c1b1c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:13.726221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0263d-3f44-432b-8528-6fc4c1b1c40a None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.726745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0263d-3f44-432b-8528-6fc4c1b1c40a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.727259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0263d-3f44-432b-8528-6fc4c1b1c40a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.728233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0263d-3f44-432b-8528-6fc4c1b1c40a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:13.730109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0263d-3f44-432b-8528-6fc4c1b1c40a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:13.742328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22522256-f5b0-4d0f-8a6e-0de7d074369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:13.745571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22522256-f5b0-4d0f-8a6e-0de7d074369a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.745571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22522256-f5b0-4d0f-8a6e-0de7d074369a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.745571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22522256-f5b0-4d0f-8a6e-0de7d074369a None admin] PATH_INFO: `/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.745833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22522256-f5b0-4d0f-8a6e-0de7d074369a None admin] RBAC: Authorizing `identity:get_user(user_id=7e2685b786dd4d2eb4cf48fedfc3b189)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:13.747815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22522256-f5b0-4d0f-8a6e-0de7d074369a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:13.754445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 18/36] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:13 2022] GET /identity/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189 => generated 248 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:13.754445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349ff31-fe12-4628-aa0e-52e70f41909b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.754445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349ff31-fe12-4628-aa0e-52e70f41909b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.754445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349ff31-fe12-4628-aa0e-52e70f41909b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.754445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 19/37] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:13.755111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12edc9-5cb2-4b29-8c0a-891ebb46e48f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.755445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12edc9-5cb2-4b29-8c0a-891ebb46e48f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.756413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12edc9-5cb2-4b29-8c0a-891ebb46e48f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.757512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 20/38] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:13.758097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12edc9-5cb2-4b29-8c0a-891ebb46e48f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:13.792788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:13.840029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 19/39] 10.222.0.128 () {60 vars in 1136 bytes} [Tue Jul 26 15:15:13 2022] POST /identity/v3/roles => generated 226 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:13.841211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935772a6-8cc6-4a11-a597-f901ed70c951 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.841507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935772a6-8cc6-4a11-a597-f901ed70c951 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.841793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935772a6-8cc6-4a11-a597-f901ed70c951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.863816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-935772a6-8cc6-4a11-a597-f901ed70c951 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:13.866097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-935772a6-8cc6-4a11-a597-f901ed70c951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:13.886770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12edc9-5cb2-4b29-8c0a-891ebb46e48f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:13.897916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:13.930726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf0a42-b86e-42c6-8184-b990ce5ac84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:13.931409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0a42-b86e-42c6-8184-b990ce5ac84d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.931699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0a42-b86e-42c6-8184-b990ce5ac84d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.931983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0a42-b86e-42c6-8184-b990ce5ac84d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.933351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf0a42-b86e-42c6-8184-b990ce5ac84d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:13.934834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf0a42-b86e-42c6-8184-b990ce5ac84d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:13.936698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 21/40] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:13 2022] GET /identity/v3/domains/default => generated 200 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:13.937698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793a65e-64dc-4715-a44b-7b9befffeeae None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:13.937992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793a65e-64dc-4715-a44b-7b9befffeeae None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:13.938276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793a65e-64dc-4715-a44b-7b9befffeeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:13.958611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5793a65e-64dc-4715-a44b-7b9befffeeae None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:13.968774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 20/41] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:13.968985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5793a65e-64dc-4715-a44b-7b9befffeeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:13.969077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935772a6-8cc6-4a11-a597-f901ed70c951 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:14.055712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 22/42] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:14.056630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793a65e-64dc-4715-a44b-7b9befffeeae None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:14.082361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:14.096672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935772a6-8cc6-4a11-a597-f901ed70c951 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:14.134717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:14.135404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:14.135688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:14.135974 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] PATH_INFO: `/v3/domains/default/users/7e2685b786dd4d2eb4cf48fedfc3b189/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:14.138032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7e2685b786dd4d2eb4cf48fedfc3b189, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:14.139667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:14.159919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793a65e-64dc-4715-a44b-7b9befffeeae None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:14.195074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:14.195734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:14.196031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:14.196413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:14.197523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:14.200960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:14.200960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 23/43] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:14 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:14.201329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:14.232728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 21/44] 10.222.0.128 () {58 vars in 1366 bytes} [Tue Jul 26 15:15:13 2022] PUT /identity/v3/domains/default/users/7e2685b786dd4d2eb4cf48fedfc3b189/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:14.232728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:14.245323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:14.306982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:14.317385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa7947-64f0-4cfb-a6e5-f9a1c5d899c2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:14.341441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f06e0-ff8d-4551-acdd-113aa5f7afa7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:14.359184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c969161-e09e-47b6-a265-cf94f60cdc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:14.359833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c969161-e09e-47b6-a265-cf94f60cdc19 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:14.361121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c969161-e09e-47b6-a265-cf94f60cdc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:14.361404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c969161-e09e-47b6-a265-cf94f60cdc19 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:14.362153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c969161-e09e-47b6-a265-cf94f60cdc19 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:14.363702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c969161-e09e-47b6-a265-cf94f60cdc19 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:14.387435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbb82d-0bb8-42de-b342-c9c669f8d694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:14.388080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbb82d-0bb8-42de-b342-c9c669f8d694 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:14.388369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbb82d-0bb8-42de-b342-c9c669f8d694 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:14.388681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbb82d-0bb8-42de-b342-c9c669f8d694 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:14.389392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbb82d-0bb8-42de-b342-c9c669f8d694 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:14.390881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbb82d-0bb8-42de-b342-c9c669f8d694 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:14.463630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 24/45] 10.222.0.128 () {60 vars in 1136 bytes} [Tue Jul 26 15:15:14 2022] POST /identity/v3/roles => generated 224 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:14.493309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 22/46] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:15:14 2022] POST /identity/v3/projects => generated 313 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:16.289489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047195-5e44-4ab4-a27e-254829b9e679 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.290148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047195-5e44-4ab4-a27e-254829b9e679 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.290148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047195-5e44-4ab4-a27e-254829b9e679 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.291423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 25/47] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:16.296925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65decd-8448-4304-b55f-6d5c5739b13b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.297363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65decd-8448-4304-b55f-6d5c5739b13b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.297498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65decd-8448-4304-b55f-6d5c5739b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.316078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dd65decd-8448-4304-b55f-6d5c5739b13b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:16.317329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dd65decd-8448-4304-b55f-6d5c5739b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:16.381172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:16.385808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 23/48] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:16.409244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7047195-5e44-4ab4-a27e-254829b9e679 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:16.515677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7047195-5e44-4ab4-a27e-254829b9e679 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:16.535611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c42951-0c45-461b-8cf1-17468166eb35 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.535913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c42951-0c45-461b-8cf1-17468166eb35 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.536203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c42951-0c45-461b-8cf1-17468166eb35 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.538283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 24/49] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:16.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd17486-63fc-4c21-8b45-bccd21b15531 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.543611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd17486-63fc-4c21-8b45-bccd21b15531 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.543902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd17486-63fc-4c21-8b45-bccd21b15531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.550212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:16.550849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.551142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.551420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.562427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:16.563056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fbd17486-63fc-4c21-8b45-bccd21b15531 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:16.564270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fbd17486-63fc-4c21-8b45-bccd21b15531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:16.567347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:16.568895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:16.569787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 26/50] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:16.575542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:16.590417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:16.634064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 25/51] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:16.656656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd17486-63fc-4c21-8b45-bccd21b15531 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:16.677147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441a8e3-9e6e-4748-b7bc-ef2cacca1d42 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:16.712107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:16.712779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.713069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.713350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.714079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:16.714391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:16.715790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:16.724033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 27/52] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:16.730806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:16.759566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd17486-63fc-4c21-8b45-bccd21b15531 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:16.788086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:16.794182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:16.794781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.795071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.795358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.796550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:16.798011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:16.799701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 26/53] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:16 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:16.806095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:16.806491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:16.831346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dc2bd-79b2-4bd9-a3f6-8ecfb5100d17 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:16.865505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:16.866446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.866829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.867122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] PATH_INFO: `/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.868411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] RBAC: Authorizing `identity:get_user(user_id=7e2685b786dd4d2eb4cf48fedfc3b189)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:16.869910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:16.872187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 28/54] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:16 2022] GET /identity/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189 => generated 248 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:16.877331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:16.909355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923eb007-f18c-47cd-bfa8-4c47f39001c0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:16.945096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24fd66-59bb-45ac-9dad-de32bdf82235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:16.945729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24fd66-59bb-45ac-9dad-de32bdf82235 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:16.946013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24fd66-59bb-45ac-9dad-de32bdf82235 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:16.946287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24fd66-59bb-45ac-9dad-de32bdf82235 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:16.946970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24fd66-59bb-45ac-9dad-de32bdf82235 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:16.948427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24fd66-59bb-45ac-9dad-de32bdf82235 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:16.979714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638f5c9-3af7-4f2f-a222-5b264064364f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:17.013522 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:17.014288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:17.014575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:17.014855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:17.016011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:17.017482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:17.019053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 29/55] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:16 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:17.024865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:17.028696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 27/56] 10.222.0.128 () {60 vars in 1145 bytes} [Tue Jul 26 15:15:16 2022] POST /identity/v3/projects => generated 299 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:17.038617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:17.137698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b7cd-89f9-45b4-98f8-72826ec340a4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:17.173686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb01f6-b973-408e-bcf9-db5a46c2ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:17.174327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb01f6-b973-408e-bcf9-db5a46c2ede9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:17.174616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb01f6-b973-408e-bcf9-db5a46c2ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:17.174897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb01f6-b973-408e-bcf9-db5a46c2ede9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:17.175712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb01f6-b973-408e-bcf9-db5a46c2ede9 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.domain.id=default, user.id=7e2685b786dd4d2eb4cf48fedfc3b189) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:17.176028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb01f6-b973-408e-bcf9-db5a46c2ede9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:17.178088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb01f6-b973-408e-bcf9-db5a46c2ede9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:17.187468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 30/57] 10.222.0.128 () {56 vars in 1327 bytes} [Tue Jul 26 15:15:17 2022] GET /identity/v3/role_assignments?user.id=7e2685b786dd4d2eb4cf48fedfc3b189&scope.domain.id=default&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 532 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:19.058133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2494d-e482-4b5c-a4d5-d1090dca20ba None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:19.058680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2494d-e482-4b5c-a4d5-d1090dca20ba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:19.058680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2494d-e482-4b5c-a4d5-d1090dca20ba None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:19.060064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 28/58] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:19.065170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f06ae-6e23-437a-a9fd-11c653553fa1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:19.065554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f06ae-6e23-437a-a9fd-11c653553fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:19.065733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f06ae-6e23-437a-a9fd-11c653553fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:19.084873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a59f06ae-6e23-437a-a9fd-11c653553fa1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:19.086063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a59f06ae-6e23-437a-a9fd-11c653553fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:19.151608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 31/59] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:19.175920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2494d-e482-4b5c-a4d5-d1090dca20ba None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:19.227124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:19.277160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2494d-e482-4b5c-a4d5-d1090dca20ba None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:19.310761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522b026-d85b-4506-ba4b-2fcc3768ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:19.311341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b026-d85b-4506-ba4b-2fcc3768ffd7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:19.311629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b026-d85b-4506-ba4b-2fcc3768ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:19.311927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b026-d85b-4506-ba4b-2fcc3768ffd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:19.313140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522b026-d85b-4506-ba4b-2fcc3768ffd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:19.314606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522b026-d85b-4506-ba4b-2fcc3768ffd7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:19.316186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 29/60] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:19 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:19.321821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f06ae-6e23-437a-a9fd-11c653553fa1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:19.324895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:19.418947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f06ae-6e23-437a-a9fd-11c653553fa1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:19.452229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9e8d5-c31c-4b30-ac39-9e1798ee6e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:19.452904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9e8d5-c31c-4b30-ac39-9e1798ee6e84 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:19.453199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9e8d5-c31c-4b30-ac39-9e1798ee6e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:19.453482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9e8d5-c31c-4b30-ac39-9e1798ee6e84 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:19.454185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9e8d5-c31c-4b30-ac39-9e1798ee6e84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:19.455656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9e8d5-c31c-4b30-ac39-9e1798ee6e84 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:19.547883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 32/61] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:15:19 2022] POST /identity/v3/users => generated 276 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:22.768218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2669b-4ba9-4334-82f2-3e8d9976047e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:22.769264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2669b-4ba9-4334-82f2-3e8d9976047e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:22.769756 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2669b-4ba9-4334-82f2-3e8d9976047e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:22.772059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 30/62] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:22 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:22.777616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759c7c-780e-4ad3-8188-e7464d98df43 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:22.778575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759c7c-780e-4ad3-8188-e7464d98df43 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:22.779070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759c7c-780e-4ad3-8188-e7464d98df43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:22.809690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-76759c7c-780e-4ad3-8188-e7464d98df43 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:22.811266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-76759c7c-780e-4ad3-8188-e7464d98df43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:22.881806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789c9d3-b8b2-46b0-a422-d51d9f837fd1 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:22.882555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789c9d3-b8b2-46b0-a422-d51d9f837fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:22.883052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789c9d3-b8b2-46b0-a422-d51d9f837fd1 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:22.884665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 31/63] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:22 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:22.898024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a110a9e-dd42-4ae7-b2a2-a1cc84842cc7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:22.898314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:22.898823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a110a9e-dd42-4ae7-b2a2-a1cc84842cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:22.899308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a110a9e-dd42-4ae7-b2a2-a1cc84842cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:22.910551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 33/64] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:22.936991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759c7c-780e-4ad3-8188-e7464d98df43 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:22.948197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1a110a9e-dd42-4ae7-b2a2-a1cc84842cc7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:22.949465 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1a110a9e-dd42-4ae7-b2a2-a1cc84842cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:23.062034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759c7c-780e-4ad3-8188-e7464d98df43 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.076032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 32/65] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:23.077725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5d44b-8ec7-45e6-b210-978ad5d34fbb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.078167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5d44b-8ec7-45e6-b210-978ad5d34fbb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.078614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5d44b-8ec7-45e6-b210-978ad5d34fbb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.080027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 33/66] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:23.085904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e1934-2085-46b4-9b41-9b7537c8de7c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.086476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e1934-2085-46b4-9b41-9b7537c8de7c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.086956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e1934-2085-46b4-9b41-9b7537c8de7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.106061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-472e1934-2085-46b4-9b41-9b7537c8de7c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:23.107912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-472e1934-2085-46b4-9b41-9b7537c8de7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:23.130236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.130976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.131191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.131463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.147700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:23.155304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:23.156204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 34/67] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:23.157321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.165639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:23.192288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 34/68] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:23.196731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e1934-2085-46b4-9b41-9b7537c8de7c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.224420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:23.278836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35607a6b-0e08-4b58-ae23-7a55f299d9cc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.307222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e1934-2085-46b4-9b41-9b7537c8de7c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.314163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5acf5-2a78-4e1e-be79-c5cb3c908fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.314757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5acf5-2a78-4e1e-be79-c5cb3c908fff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.315048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5acf5-2a78-4e1e-be79-c5cb3c908fff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.315330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5acf5-2a78-4e1e-be79-c5cb3c908fff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.325626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5acf5-2a78-4e1e-be79-c5cb3c908fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1fe5acf5-2a78-4e1e-be79-c5cb3c908fff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:23.330577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:23.331556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:23.332309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 35/69] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:23.332965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3666c90-ae06-48f0-9551-b953f14ce0ef None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.334171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3666c90-ae06-48f0-9551-b953f14ce0ef None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.334171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3666c90-ae06-48f0-9551-b953f14ce0ef None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.335271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 36/70] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:23.335988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3666c90-ae06-48f0-9551-b953f14ce0ef None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.346858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ab803-d4a1-4d25-8f45-edc86a711b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.347493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ab803-d4a1-4d25-8f45-edc86a711b54 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.347781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ab803-d4a1-4d25-8f45-edc86a711b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.348058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ab803-d4a1-4d25-8f45-edc86a711b54 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.348818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ab803-d4a1-4d25-8f45-edc86a711b54 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:23.350038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ab803-d4a1-4d25-8f45-edc86a711b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:23.351598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ab803-d4a1-4d25-8f45-edc86a711b54 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:23.360383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 35/71] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:23.361793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030728-8d31-403e-8b0c-9f0e0d9c4970 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.362093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030728-8d31-403e-8b0c-9f0e0d9c4970 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.362376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030728-8d31-403e-8b0c-9f0e0d9c4970 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.363595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 36/72] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:23 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:23.364301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030728-8d31-403e-8b0c-9f0e0d9c4970 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.408409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:23.446600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3666c90-ae06-48f0-9551-b953f14ce0ef None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.457067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:23.472868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030728-8d31-403e-8b0c-9f0e0d9c4970 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.480794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dce87-507b-4eb4-8bdd-044ec7968964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.481444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dce87-507b-4eb4-8bdd-044ec7968964 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.481732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dce87-507b-4eb4-8bdd-044ec7968964 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.482015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dce87-507b-4eb4-8bdd-044ec7968964 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.483197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dce87-507b-4eb4-8bdd-044ec7968964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:23.484698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dce87-507b-4eb4-8bdd-044ec7968964 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:23.486292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 37/73] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/domains/default => generated 200 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:23.487478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea15e8-f449-4ad4-b0bf-ff241ea8293f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.487778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea15e8-f449-4ad4-b0bf-ff241ea8293f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.488064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea15e8-f449-4ad4-b0bf-ff241ea8293f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.507590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5bea15e8-f449-4ad4-b0bf-ff241ea8293f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:23.508935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5bea15e8-f449-4ad4-b0bf-ff241ea8293f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:23.512125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.512768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.513054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.513332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.514053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:23.514361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:23.515767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:23.524293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 37/74] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:23.525122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.592536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 38/75] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:23.592536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4d42e-ab2b-464e-9d1d-9524445c4d17 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.592536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4d42e-ab2b-464e-9d1d-9524445c4d17 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.592536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4d42e-ab2b-464e-9d1d-9524445c4d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.598501 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c4e4d42e-ab2b-464e-9d1d-9524445c4d17 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:23.599928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c4e4d42e-ab2b-464e-9d1d-9524445c4d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:23.628621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd26d5-b494-4258-adea-93a2b9e2f0c0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.630227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:23.663711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.664401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.665126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.665401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.669166 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 39/76] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:23.669996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4d42e-ab2b-464e-9d1d-9524445c4d17 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.687984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:23.689656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:23.691560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 38/77] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:23.692212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.711531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:23.771663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4d42e-ab2b-464e-9d1d-9524445c4d17 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.794357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28018b-2852-485d-87ad-aa400ac3ab1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.806814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.807443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.807736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.808015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.808748 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:23.810203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:23.828025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.828700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.828986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.829262 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] PATH_INFO: `/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.830432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] RBAC: Authorizing `identity:get_user(user_id=7e2685b786dd4d2eb4cf48fedfc3b189)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:23.831864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:23.833975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 39/78] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189 => generated 248 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:23.834518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.862520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:23.890915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 40/79] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:15:23 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:23.890915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:23.941851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362b09e-f538-4933-b1c9-86082bf5dbb5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:23.982384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:23.983070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:23.983355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:23.983626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:23.997027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:24.003368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:24.004304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 40/80] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:24.011374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.018704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.027777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6164e2-cdc0-405e-809d-143768d66819 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.068359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.069069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.069357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.069633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.081776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:24.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:24.088374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:24.089190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 41/81] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:24.089817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.123047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.138678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48138-a8fc-4a8d-958c-6a603d1b42b2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.189484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.190070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.190352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.194946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.238919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d2d1-a383-4e5b-9ef1-6463bb742b36 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.289090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.290165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.290454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.290736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.291107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:24.292811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:24.324207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.390433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:24.392238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:24.400151 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.416170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 41/82] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:24.417544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.526117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 42/83] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:24.527071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.550038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248d5b3-3ac8-4a05-820f-825e473ba0f8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.592939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.593604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.593893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.594175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.595120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:24.595224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:24.596711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:24.598304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.606208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 42/84] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:24.607056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.660216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d812-b693-4ae7-976e-520d6d3018e5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.668866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.709226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.710223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.710744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.711275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.712197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:24.712721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:24.714282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:24.715669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed73f56-67c5-4785-9601-06dcbb4413e0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.723440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 43/85] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:24.726020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.759434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.760555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.760761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.760978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.762206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:24.763692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:24.765394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 43/86] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:24.766164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.857102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f3f1-c07d-4f8f-b489-548ea913956c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.876109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.879529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6685f-41c5-4e94-aaaf-7160e75b8b8e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:24.896607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.897683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.898344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:24.898840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.899337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.900770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:24.902788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:24.904714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 44/87] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:24.905667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:24.915880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:24.916473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:24.916788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:24.917072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:24.918381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:24.919914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:24.921646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 44/88] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:24.922406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:25.018594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e244e-b187-4236-a89d-3d071afcedc0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:25.035067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e011f9a-e05c-4583-9636-27a584ccad38 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:25.056091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:25.056789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:25.057082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:25.057373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:25.058684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:25.060161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:25.062031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 45/89] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:25.062811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:25.079174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:25.079934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:25.080227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:25.080526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:25.081332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:25.081647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:25.083382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:25.089385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 45/90] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:24 2022] GET /identity/v3/role_assignments?user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:25.097097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:25.125327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:25.182256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141da32-5f74-43d6-ae46-005954879143 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:25.194445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:25.223346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:25.224563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:25.224918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:25.225208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:25.226087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=7e2685b786dd4d2eb4cf48fedfc3b189) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:25.226406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:25.228055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:25.236083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 46/91] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:25 2022] GET /identity/v3/role_assignments?user.id=7e2685b786dd4d2eb4cf48fedfc3b189&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:25.242205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:25.329616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad71be3-e8bd-4a50-aa51-1770f6267cc8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:25.422225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:25.423224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:25.423733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:25.424219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:25.537656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc9f6-2645-41d5-8a00-b7d6df24f621 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:25.540146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:25.541896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:25.575418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:25.596632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:25.597278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:25.597573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:25.597860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:25.628740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:25.720723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:25.722143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:25.807170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 46/92] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:25.817616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:25.994181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 47/93] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:25.999692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:26.060489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:26.088892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9499a-3553-4c14-9ff4-cfd1660dbbf3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:26.169190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:26.170279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.170789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.171283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] PATH_INFO: `/v3/projects/c22837fa2021499b9127291e5d53069d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.177352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] RBAC: Authorizing `identity:get_project(project_id=c22837fa2021499b9127291e5d53069d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:26.179107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:26.181068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 47/94] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:25 2022] GET /identity/v3/projects/c22837fa2021499b9127291e5d53069d => generated 313 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:26.202693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:26.221759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7952558-2253-48de-9315-0ffc6057c9fd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:26.256619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:26.337232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:26.337913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.338207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.338498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.340103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:26.341345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:26.343033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 48/95] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:25 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:26.349566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:26.350268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677f46e-41a9-4cef-85c8-52ac3601207f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:26.385533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb115eb-3056-4a93-9eb0-189625c30d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:26.386517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb115eb-3056-4a93-9eb0-189625c30d2e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.386994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb115eb-3056-4a93-9eb0-189625c30d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.387487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb115eb-3056-4a93-9eb0-189625c30d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.388493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb115eb-3056-4a93-9eb0-189625c30d2e None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=c22837fa2021499b9127291e5d53069d, user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:26.389005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb115eb-3056-4a93-9eb0-189625c30d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:26.390674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb115eb-3056-4a93-9eb0-189625c30d2e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:26.397396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 48/96] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:26 2022] GET /identity/v3/role_assignments?user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c&scope.project.id=c22837fa2021499b9127291e5d53069d&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:26.497853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a139f64-4dd4-4deb-ab9e-1508cf1d0db2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.498470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a139f64-4dd4-4deb-ab9e-1508cf1d0db2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.498980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a139f64-4dd4-4deb-ab9e-1508cf1d0db2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.500533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 49/97] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:26.508941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:26.510325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410442f1-d014-4539-931b-d8a96cc3abcf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.510809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410442f1-d014-4539-931b-d8a96cc3abcf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.511276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410442f1-d014-4539-931b-d8a96cc3abcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.524645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc9b73-f1a4-4663-9510-4f67d891f9e4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:26.565925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:26.566586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.566869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-410442f1-d014-4539-931b-d8a96cc3abcf None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:26.566970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.567159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.568623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] RBAC: Adding query filter params (role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:26.568950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:26.569589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-410442f1-d014-4539-931b-d8a96cc3abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:26.570433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:26.581599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 49/98] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:26 2022] GET /identity/v3/role_assignments?user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:26.722596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 50/99] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:26.762566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:26.780044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742ed37-b72b-4db0-ba07-09a23bd076ca None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.780365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742ed37-b72b-4db0-ba07-09a23bd076ca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.780648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742ed37-b72b-4db0-ba07-09a23bd076ca None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.782153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 51/100] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:26.788434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d5ae-a9ae-4af9-b970-4d6bd450d4bf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.788757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d5ae-a9ae-4af9-b970-4d6bd450d4bf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.789097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d5ae-a9ae-4af9-b970-4d6bd450d4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.825765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c129d5ae-a9ae-4af9-b970-4d6bd450d4bf None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:26.826995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c129d5ae-a9ae-4af9-b970-4d6bd450d4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:26.835341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:26.877541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99234431-3770-44ae-aabd-d7a9e894aa2d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:26.916721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:26.917383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:26.917677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:26.917962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:26.919888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:26.921393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:26.923020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 50/101] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:26 2022] GET /identity/v3/domains/default => generated 200 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:26.936848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:26.947591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:26.975114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 52/102] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:27.015580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129d5ae-a9ae-4af9-b970-4d6bd450d4bf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:27.036986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16db2-69a4-40fc-a964-9e2131b733b0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:27.081116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:27.082018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:27.082472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:27.082899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:27.084294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:27.086393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:27.162963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129d5ae-a9ae-4af9-b970-4d6bd450d4bf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:27.203259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:27.204176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:27.204671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:27.205127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:27.216539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:27.221823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:27.222935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:27.223986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:27.226113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 53/103] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:27.232713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:27.248063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:27.268620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 51/104] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:15:26 2022] POST /identity/v3/projects => generated 303 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:15:27.361322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014b20-5240-4b08-a623-68ffc88eb322 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:27.400536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:27.401452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:27.401910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:27.402344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:27.403241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:27.403708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:27.405335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:27.415513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 54/105] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:27 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:27.421290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:27.558775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:27.605196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2de36d-2a67-46c2-9b87-a619dd979c2d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:27.655635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cf4fe-b79b-4a29-aeeb-4307bd9e1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:27.656297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cf4fe-b79b-4a29-aeeb-4307bd9e1fef None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:27.656646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cf4fe-b79b-4a29-aeeb-4307bd9e1fef None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:27.656936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cf4fe-b79b-4a29-aeeb-4307bd9e1fef None admin] PATH_INFO: `/v3/users/5fd2d30736e443fcbba05e30baa54e7c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:27.681441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cf4fe-b79b-4a29-aeeb-4307bd9e1fef None admin] RBAC: Authorizing `identity:get_user(user_id=5fd2d30736e443fcbba05e30baa54e7c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:27.683305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cf4fe-b79b-4a29-aeeb-4307bd9e1fef None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:27.685294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 52/106] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:27 2022] GET /identity/v3/users/5fd2d30736e443fcbba05e30baa54e7c => generated 290 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:27.690822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:27.783238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:27.798055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f80d7-dc54-4283-9b23-ab25b26453de None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:27.801204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f80d7-dc54-4283-9b23-ab25b26453de None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:27.801204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f80d7-dc54-4283-9b23-ab25b26453de None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:27.801204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 53/107] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:27.806961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf41d6c-0c1e-4d24-aa3f-783ac44f3174 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:27.807484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf41d6c-0c1e-4d24-aa3f-783ac44f3174 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:27.807965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf41d6c-0c1e-4d24-aa3f-783ac44f3174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:27.826139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856ada-9eb1-4430-b627-29bf8e8c2dae None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:27.853960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2bf41d6c-0c1e-4d24-aa3f-783ac44f3174 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:27.855508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2bf41d6c-0c1e-4d24-aa3f-783ac44f3174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:27.868499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:27.868499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:27.868499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:27.868499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:27.949627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:27.951128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:28.045824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 54/108] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:28.067758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:28.091451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf41d6c-0c1e-4d24-aa3f-783ac44f3174 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:28.132047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 55/109] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:27 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:28.137483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:28.234765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:28.344420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf41d6c-0c1e-4d24-aa3f-783ac44f3174 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:28.364419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d79289-9f62-4559-a9ed-24275555060c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:28.424325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74052eb2-8682-45b4-a1a5-974a0e404146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:28.425088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052eb2-8682-45b4-a1a5-974a0e404146 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.425386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052eb2-8682-45b4-a1a5-974a0e404146 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.425673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052eb2-8682-45b4-a1a5-974a0e404146 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.427159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74052eb2-8682-45b4-a1a5-974a0e404146 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:28.428686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74052eb2-8682-45b4-a1a5-974a0e404146 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:28.433408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 56/110] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:28 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:28.434741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b92995-4bc5-4206-8ae1-26d9345a590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:28.436547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79408e-369c-464b-acdb-d926e8e2daeb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.437057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92995-4bc5-4206-8ae1-26d9345a590c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.437485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92995-4bc5-4206-8ae1-26d9345a590c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.437916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92995-4bc5-4206-8ae1-26d9345a590c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.438863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79408e-369c-464b-acdb-d926e8e2daeb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.439505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79408e-369c-464b-acdb-d926e8e2daeb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.444048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 57/111] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:28 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:28.449430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79408e-369c-464b-acdb-d926e8e2daeb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:28.474297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b92995-4bc5-4206-8ae1-26d9345a590c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-72b92995-4bc5-4206-8ae1-26d9345a590c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:28.479299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:28.480168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:28.480982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:28.488137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:28.488397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 55/112] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:28.490512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1fd47-b4a8-4bc4-b8a8-f82d5208b7d5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.490828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1fd47-b4a8-4bc4-b8a8-f82d5208b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.491117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1fd47-b4a8-4bc4-b8a8-f82d5208b7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.529208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a3d1fd47-b4a8-4bc4-b8a8-f82d5208b7d5 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:28.532313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a3d1fd47-b4a8-4bc4-b8a8-f82d5208b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:28.593416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79408e-369c-464b-acdb-d926e8e2daeb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:28.637447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:28.640015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.640545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.641030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.644393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] RBAC: Adding query filter params (role.id=a9b476e637814e0f8b3a8aa8e5ca9578, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=5fd2d30736e443fcbba05e30baa54e7c) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:28.644953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:28.647352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:28.659188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 58/113] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:28 2022] GET /identity/v3/role_assignments?user.id=5fd2d30736e443fcbba05e30baa54e7c&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=a9b476e637814e0f8b3a8aa8e5ca9578 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:28.660204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:28.671100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 56/114] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:28.695618 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1fd47-b4a8-4bc4-b8a8-f82d5208b7d5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:28.749482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:28.812100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1fd47-b4a8-4bc4-b8a8-f82d5208b7d5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:28.850678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:28.851724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.852247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.852775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.866422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:28.873911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:28.874828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:28.875601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:28.878406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 57/115] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:28.885376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:28.904775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:28.913579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fad026-07d7-45c1-8379-e7d9298bc33d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:28.981230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:28.982264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.982765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.983258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.984217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:28.984776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:28.986387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.008325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 59/116] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:28 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:29.010377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7862c8-4574-4eeb-9f03-4e3945ce08db None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:29.014885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:29.024897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:29.054575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:29.056019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.056570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.057075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.058018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:29.058533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:29.060130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.069772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 58/117] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:28 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:29.076200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:29.184166 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01694a4-76e9-419f-a154-2b1172e8ce39 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:29.223975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:29.225035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.225530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.226048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] PATH_INFO: `/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.227495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] RBAC: Authorizing `identity:get_user(user_id=7e2685b786dd4d2eb4cf48fedfc3b189)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:29.229174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.231019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 59/118] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:29 2022] GET /identity/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189 => generated 248 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:29.236565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:29.263708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24be26-f154-4d7a-a2b5-25e26e19cba5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:29.285284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:29.353117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:29.353772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.354054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.354328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.356328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:29.362118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.363887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 60/119] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:29 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:29.373492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:29.430082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:29.431570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b49a05-5629-42fe-9296-540e9cc55778 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:29.530956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:29.531617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.531904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.532184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.537241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:29.538723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.540425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 60/120] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:29 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:29.546210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:29.592805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead0544-a374-46d4-8a0a-50ec2a47f3f7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:29.649256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f6b80-e465-401d-baa9-f49e9ca22ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:29.649924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f6b80-e465-401d-baa9-f49e9ca22ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.650222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f6b80-e465-401d-baa9-f49e9ca22ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.650505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f6b80-e465-401d-baa9-f49e9ca22ab6 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.657249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f6b80-e465-401d-baa9-f49e9ca22ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:29.658752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f6b80-e465-401d-baa9-f49e9ca22ab6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.661032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 61/121] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:29 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:29.662320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0aed9-9176-4923-b522-f2664b59d835 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.662620 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0aed9-9176-4923-b522-f2664b59d835 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.662908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0aed9-9176-4923-b522-f2664b59d835 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.664226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 62/122] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:29.668101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0aed9-9176-4923-b522-f2664b59d835 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:29.688950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1a7f-3097-4674-b6ae-a4322ffeeef3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:29.692978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:29.762264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbffa4e-8828-45e8-8901-d68abf5ed14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:29.763059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbffa4e-8828-45e8-8901-d68abf5ed14a None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.763461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbffa4e-8828-45e8-8901-d68abf5ed14a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.763976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbffa4e-8828-45e8-8901-d68abf5ed14a None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/7e2685b786dd4d2eb4cf48fedfc3b189/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.769792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbffa4e-8828-45e8-8901-d68abf5ed14a None admin] RBAC: Authorizing `identity:create_grant(project_id=fec0664d0d794aee9dc2952586baef2b, user_id=7e2685b786dd4d2eb4cf48fedfc3b189, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:29.771790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbffa4e-8828-45e8-8901-d68abf5ed14a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.775444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:29.784015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0aed9-9176-4923-b522-f2664b59d835 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:29.821634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd453c2-39f2-4398-9c26-e8461f920ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:29.822576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd453c2-39f2-4398-9c26-e8461f920ea6 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.823027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd453c2-39f2-4398-9c26-e8461f920ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.823544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd453c2-39f2-4398-9c26-e8461f920ea6 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.825880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd453c2-39f2-4398-9c26-e8461f920ea6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fec0664d0d794aee9dc2952586baef2b, user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:29.827560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd453c2-39f2-4398-9c26-e8461f920ea6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:29.881529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 61/123] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:29 2022] PUT /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/7e2685b786dd4d2eb4cf48fedfc3b189/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:15:29.890247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c5b3-a57c-44ad-96af-22519a397fd9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.890612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c5b3-a57c-44ad-96af-22519a397fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.890971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c5b3-a57c-44ad-96af-22519a397fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.909456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 63/124] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:29 2022] PUT /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:15:29.911781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a16db4-1c00-4594-be82-519e68c2b21f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.912854 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a16db4-1c00-4594-be82-519e68c2b21f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.913352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a16db4-1c00-4594-be82-519e68c2b21f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.915551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 64/125] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:29 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:29.923386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1b11c5b3-a57c-44ad-96af-22519a397fd9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:29.924259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e32b1-e2cd-483a-8531-6d75bc801c27 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:29.924649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1b11c5b3-a57c-44ad-96af-22519a397fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:29.925037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e32b1-e2cd-483a-8531-6d75bc801c27 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:29.925518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e32b1-e2cd-483a-8531-6d75bc801c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:29.950535 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fe4e32b1-e2cd-483a-8531-6d75bc801c27 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:29.952270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe4e32b1-e2cd-483a-8531-6d75bc801c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:30.045798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 65/126] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:30.088306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e32b1-e2cd-483a-8531-6d75bc801c27 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:30.111480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 62/127] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:30.153933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c5b3-a57c-44ad-96af-22519a397fd9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:30.211484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:30.236024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e32b1-e2cd-483a-8531-6d75bc801c27 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:30.305382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:30.305982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:30.306279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:30.306564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:30.325432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:30.337712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c5b3-a57c-44ad-96af-22519a397fd9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:30.341602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:30.346838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:30.347764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:30.349185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 66/128] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:30.356144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:30.408517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:30.409413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:30.409848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:30.410270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:30.437548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:30.443398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:30.444297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:30.445104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:30.447283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 63/129] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:30.453128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:30.565391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9c8f4-4036-4ed0-bfb8-6ffad3271a87 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:30.625747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:30.626439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:30.626658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:30.626941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:30.632367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:30.632367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:30.632367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:30.643206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 67/130] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:15:30 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:30.650398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:30.656764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096b756-9229-49cc-b922-04318a2270d5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:30.668980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:30.769205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026d84f-c5eb-4ca5-bdd5-4dff622243f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:30.769852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d84f-c5eb-4ca5-bdd5-4dff622243f0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:30.770146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d84f-c5eb-4ca5-bdd5-4dff622243f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:30.771073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d84f-c5eb-4ca5-bdd5-4dff622243f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:30.772048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026d84f-c5eb-4ca5-bdd5-4dff622243f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:30.772372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026d84f-c5eb-4ca5-bdd5-4dff622243f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:30.773662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9354fb-0e9c-434f-ba2b-cbd3f593ac1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:30.773847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026d84f-c5eb-4ca5-bdd5-4dff622243f0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:30.791855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 64/131] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:30 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:30.793180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c96d42-af83-4d6b-830c-ea82726d1028 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:30.793476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c96d42-af83-4d6b-830c-ea82726d1028 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:30.793760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c96d42-af83-4d6b-830c-ea82726d1028 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:30.795116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 65/132] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:30.801357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c96d42-af83-4d6b-830c-ea82726d1028 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:30.811072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2b818-d4d2-4654-a7b5-ebd9f6045c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:30.811734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2b818-d4d2-4654-a7b5-ebd9f6045c08 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:30.812024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2b818-d4d2-4654-a7b5-ebd9f6045c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:30.812313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2b818-d4d2-4654-a7b5-ebd9f6045c08 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:30.813534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2b818-d4d2-4654-a7b5-ebd9f6045c08 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:30.815002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2b818-d4d2-4654-a7b5-ebd9f6045c08 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:30.816686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 68/133] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:30 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:30.822804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a63a0-e22b-493f-b2b9-7e6e4efb3485 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:30.823101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a63a0-e22b-493f-b2b9-7e6e4efb3485 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:30.823390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a63a0-e22b-493f-b2b9-7e6e4efb3485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:30.842618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-177a63a0-e22b-493f-b2b9-7e6e4efb3485 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:30.843819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-177a63a0-e22b-493f-b2b9-7e6e4efb3485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:30.854414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:30.923600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 69/134] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:30.924651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a63a0-e22b-493f-b2b9-7e6e4efb3485 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:30.929045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:31.038396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a63a0-e22b-493f-b2b9-7e6e4efb3485 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:31.046177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c96d42-af83-4d6b-830c-ea82726d1028 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:31.075824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:31.076399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.076713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.076993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.078285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:31.079881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:31.081582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 70/135] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:30 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:31.087174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:31.113833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:31.115140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.115365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.116096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.121475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:31.122929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:31.124591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 66/136] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:30 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:31.125348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:31.164067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:31.203127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca5cb6-8cbf-4993-b25e-c1e9b7844d5d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:31.249608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:31.250268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.250554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.250832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.251986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:31.253631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:31.255481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 71/137] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:31 2022] GET /identity/v3/domains/default => generated 200 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:31.256325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:31.319068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:31.364755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5966d-d2ad-40a7-b7a0-87eaee4714b5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:31.367313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c79142-bb90-482d-92d5-bb8f4aa49e3f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:31.403742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:31.404395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.404716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.404997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] PATH_INFO: `/v3/projects/c22837fa2021499b9127291e5d53069d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.406156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] RBAC: Authorizing `identity:get_project(project_id=c22837fa2021499b9127291e5d53069d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:31.407619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:31.409639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 72/138] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:31 2022] GET /identity/v3/projects/c22837fa2021499b9127291e5d53069d => generated 313 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:31.410499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:31.422518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:31.426655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2760b2-a930-4b0d-86c9-ce8ff4db0e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:31.427753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2760b2-a930-4b0d-86c9-ce8ff4db0e05 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.428718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2760b2-a930-4b0d-86c9-ce8ff4db0e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.429191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2760b2-a930-4b0d-86c9-ce8ff4db0e05 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.432281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2760b2-a930-4b0d-86c9-ce8ff4db0e05 None admin] RBAC: Authorizing `identity:create_grant(project_id=fec0664d0d794aee9dc2952586baef2b, user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c, role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:31.433941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2760b2-a930-4b0d-86c9-ce8ff4db0e05 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:31.521712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c5f41-a579-445a-80c0-61fdcff039e3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:31.545647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 67/139] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:31 2022] PUT /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c/roles/364a95d0748e48e9a538b4e6be293f91 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:31.547311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dda8c-ebf4-40e2-a6b7-66d3204024c7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.547762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dda8c-ebf4-40e2-a6b7-66d3204024c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.548273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dda8c-ebf4-40e2-a6b7-66d3204024c7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.550022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 68/140] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:31.555910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dda8c-ebf4-40e2-a6b7-66d3204024c7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:31.589017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35381-0911-4d17-a385-9811cc36febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:31.589664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35381-0911-4d17-a385-9811cc36febf None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.589948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35381-0911-4d17-a385-9811cc36febf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.590290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35381-0911-4d17-a385-9811cc36febf None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.590964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35381-0911-4d17-a385-9811cc36febf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:31.592490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35381-0911-4d17-a385-9811cc36febf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:31.693220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dda8c-ebf4-40e2-a6b7-66d3204024c7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:31.734589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:31.735691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.736172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.736758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] PATH_INFO: `/v3/projects/c22837fa2021499b9127291e5d53069d/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.739126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] RBAC: Authorizing `identity:create_grant(project_id=c22837fa2021499b9127291e5d53069d, user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:31.741047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:31.743166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 73/141] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:15:31 2022] POST /identity/v3/users => generated 284 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:31.748612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337ad52-bcf9-46e6-831d-657501a5d12f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:31.748612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337ad52-bcf9-46e6-831d-657501a5d12f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:31.748612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337ad52-bcf9-46e6-831d-657501a5d12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:31.777448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5337ad52-bcf9-46e6-831d-657501a5d12f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:31.778671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5337ad52-bcf9-46e6-831d-657501a5d12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:31.848691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 69/142] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:31 2022] PUT /identity/v3/projects/c22837fa2021499b9127291e5d53069d/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:31.855552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:31.864994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 74/143] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:31.910393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:31.915471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:32.056639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa1b55-61cb-4d4b-b9d4-1069ffc80c3b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:32.096236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:32.097287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.097832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.099236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.110732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:32.116555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:32.117845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:32.118577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:32.121518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 70/144] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:32.130580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337ad52-bcf9-46e6-831d-657501a5d12f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:32.266297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337ad52-bcf9-46e6-831d-657501a5d12f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:32.308084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954554da-fec4-446e-ad09-28736ef1a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:32.308780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954554da-fec4-446e-ad09-28736ef1a8de None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.309068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954554da-fec4-446e-ad09-28736ef1a8de None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.309346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954554da-fec4-446e-ad09-28736ef1a8de None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.310153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954554da-fec4-446e-ad09-28736ef1a8de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:32.310461 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954554da-fec4-446e-ad09-28736ef1a8de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:32.311855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954554da-fec4-446e-ad09-28736ef1a8de None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:32.321829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 75/145] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:32 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:32.327729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:32.425860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:32.482054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb202e7e-c0e0-4276-9cef-7fbf5ea849d0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:32.528388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a76dce-be51-4109-b7ce-7c78146e6114 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.528613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e9fe9-ad3a-405f-b90b-6e56bf5652c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:32.529174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a76dce-be51-4109-b7ce-7c78146e6114 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.529273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9fe9-ad3a-405f-b90b-6e56bf5652c0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.529478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9fe9-ad3a-405f-b90b-6e56bf5652c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.529767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9fe9-ad3a-405f-b90b-6e56bf5652c0 None admin] PATH_INFO: `/v3/users/5fd2d30736e443fcbba05e30baa54e7c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.530144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a76dce-be51-4109-b7ce-7c78146e6114 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.531407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e9fe9-ad3a-405f-b90b-6e56bf5652c0 None admin] RBAC: Authorizing `identity:get_user(user_id=5fd2d30736e443fcbba05e30baa54e7c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:32.531908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 76/146] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:32 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:32.534741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e9fe9-ad3a-405f-b90b-6e56bf5652c0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:32.534741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 71/147] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:32 2022] GET /identity/v3/users/5fd2d30736e443fcbba05e30baa54e7c => generated 290 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:32.539924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b219ff6-8b36-417e-a065-c42d283f35d3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.540229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b219ff6-8b36-417e-a065-c42d283f35d3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.540543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b219ff6-8b36-417e-a065-c42d283f35d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.540917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a76dce-be51-4109-b7ce-7c78146e6114 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:32.574127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4b219ff6-8b36-417e-a065-c42d283f35d3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:32.575255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4b219ff6-8b36-417e-a065-c42d283f35d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:32.657008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:32.671124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 72/148] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:32.697783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b219ff6-8b36-417e-a065-c42d283f35d3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:32.735858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a76dce-be51-4109-b7ce-7c78146e6114 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:32.769998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:32.812050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b219ff6-8b36-417e-a065-c42d283f35d3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:32.818130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445970f-5ad6-4265-af41-72db1f79c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:32.818806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445970f-5ad6-4265-af41-72db1f79c688 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.819095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445970f-5ad6-4265-af41-72db1f79c688 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.819375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445970f-5ad6-4265-af41-72db1f79c688 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.820629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445970f-5ad6-4265-af41-72db1f79c688 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:32.822103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445970f-5ad6-4265-af41-72db1f79c688 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:32.824017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 77/149] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:32 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:32.829798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d62a-8b09-435e-8063-a7cf53590f63 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.829927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d62a-8b09-435e-8063-a7cf53590f63 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.830213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d62a-8b09-435e-8063-a7cf53590f63 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.831711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 78/150] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:32 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:32.832273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067d62a-8b09-435e-8063-a7cf53590f63 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:32.850202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c76882-5e5b-462e-a3ec-5241e159e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:32.850866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c76882-5e5b-462e-a3ec-5241e159e127 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.851152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c76882-5e5b-462e-a3ec-5241e159e127 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.851432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c76882-5e5b-462e-a3ec-5241e159e127 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.862604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c76882-5e5b-462e-a3ec-5241e159e127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-28c76882-5e5b-462e-a3ec-5241e159e127 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:32.868547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:32.869445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:32.871260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 73/151] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:32.874741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17bd92-49e2-44d6-a7d1-63c5896f08ec None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.874741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17bd92-49e2-44d6-a7d1-63c5896f08ec None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.874741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17bd92-49e2-44d6-a7d1-63c5896f08ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.893568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-da17bd92-49e2-44d6-a7d1-63c5896f08ec None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:32.895572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da17bd92-49e2-44d6-a7d1-63c5896f08ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:32.976676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 74/152] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:32.976676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17bd92-49e2-44d6-a7d1-63c5896f08ec None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:33.034285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067d62a-8b09-435e-8063-a7cf53590f63 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:33.036753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:33.071292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:33.091020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:33.091774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:33.092126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:33.092415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/5fd2d30736e443fcbba05e30baa54e7c/roles/a9b476e637814e0f8b3a8aa8e5ca9578` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:33.094636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fec0664d0d794aee9dc2952586baef2b, user_id=5fd2d30736e443fcbba05e30baa54e7c, role_id=a9b476e637814e0f8b3a8aa8e5ca9578)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:33.096544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:33.119841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17bd92-49e2-44d6-a7d1-63c5896f08ec None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:33.171341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:33.171983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:33.172368 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:33.172693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:33.173429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:33.173742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:33.175254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:33.183414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 79/153] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:32 2022] PUT /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b/users/5fd2d30736e443fcbba05e30baa54e7c/roles/a9b476e637814e0f8b3a8aa8e5ca9578 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:33.184269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:33.189709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 75/154] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:33.196746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:33.363719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb180-c819-4028-9906-1dfb0018dc62 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:33.400425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:33.451759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb2b4b-e35e-4c2a-91e2-fc47a512fea3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:33.455504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:33.456186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:33.456568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:33.456858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] PATH_INFO: `/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:33.460383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] RBAC: Authorizing `identity:get_user(user_id=7e2685b786dd4d2eb4cf48fedfc3b189)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:33.461931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:33.463626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 76/155] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:33 2022] GET /identity/v3/users/7e2685b786dd4d2eb4cf48fedfc3b189 => generated 248 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:33.469061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:33.519519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:33.522305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:33.522831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:33.523523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:33.548068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:33.555426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:33.556531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:33.557484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:33.560042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 80/156] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:33.565979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:33.657311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:33.666148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4051c-38bd-42a5-832c-dbef08803fae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:33.725664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:33.726656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:33.726948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:33.727234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:33.764705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3668a5-cff1-4cb9-9049-0376d4bb5782 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:33.811576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:33.837242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:33.838256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:33.838603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:33.838916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:33.839376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:33.840349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:33.840487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:33.840965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:33.842688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:33.860586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 81/157] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:33 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:33.867642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:34.036777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 77/158] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:33 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:15:34.042841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:34.086617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961d005-1e8f-4815-963a-48352fd6fc84 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:34.098351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:34.165759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:34.166408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:34.166679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:34.166954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:34.173194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:34.174728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:34.176425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 82/159] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:33 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:34.183521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:34.212841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:34.257000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9456b-169c-4790-9760-d644382c9d67 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:34.345305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:34.346206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:34.346641 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:34.347066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:34.349127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:34.351016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:34.353111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 78/160] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:34 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:34.359039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:34.432918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ff582-fe85-4f8b-8d07-71f031096a17 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:34.507468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee49f90-785e-48a3-965c-916587a96ded None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:34.545716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:34.546589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:34.547024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:34.547447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:34.548420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=7e2685b786dd4d2eb4cf48fedfc3b189) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:34.548905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:34.550531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:34.557729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:34.558024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 79/161] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:34 2022] GET /identity/v3/role_assignments?user.id=7e2685b786dd4d2eb4cf48fedfc3b189&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:34.560622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:34.560622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:34.560622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:34.634953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:34.669536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:34.671035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:34.816345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 83/162] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:15:34.823463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:34.849356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:34.965620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4bed-3fe1-459f-baf5-52b1968d65dc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:35.011079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08911671-653a-4148-b769-724c9d94a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:35.012107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08911671-653a-4148-b769-724c9d94a3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:35.012664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08911671-653a-4148-b769-724c9d94a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:35.013150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08911671-653a-4148-b769-724c9d94a3b4 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:35.014543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08911671-653a-4148-b769-724c9d94a3b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:35.016237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08911671-653a-4148-b769-724c9d94a3b4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:35.018445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 80/163] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:34 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:35.025511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:35.143560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:35.180206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8875bd-e8c1-4c0e-b548-c4736655de2b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:35.233500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e353f-4e4d-4690-afe1-b3cee6b0840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:35.234108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e353f-4e4d-4690-afe1-b3cee6b0840b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:35.234389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e353f-4e4d-4690-afe1-b3cee6b0840b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:35.234660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e353f-4e4d-4690-afe1-b3cee6b0840b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:35.235461 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e353f-4e4d-4690-afe1-b3cee6b0840b None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:35.235773 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e353f-4e4d-4690-afe1-b3cee6b0840b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:35.237229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e353f-4e4d-4690-afe1-b3cee6b0840b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:35.246863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 84/164] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:35 2022] GET /identity/v3/role_assignments?user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 610 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:35.704329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53442836-85c4-40d6-8267-7ac418b2bdf0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:35.705354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53442836-85c4-40d6-8267-7ac418b2bdf0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:35.705867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53442836-85c4-40d6-8267-7ac418b2bdf0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:35.707449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 81/165] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:35 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:35.713347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69d71-7a50-4377-a773-ad49bf2bc9c3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:35.714216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69d71-7a50-4377-a773-ad49bf2bc9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:35.714700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69d71-7a50-4377-a773-ad49bf2bc9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:35.740632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-14d69d71-7a50-4377-a773-ad49bf2bc9c3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:35.742335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-14d69d71-7a50-4377-a773-ad49bf2bc9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:35.822748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 85/166] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:35.860714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53442836-85c4-40d6-8267-7ac418b2bdf0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:35.971485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:35.996563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556de4-a3fc-4c2b-ba87-472663320bde None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:35.996778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53442836-85c4-40d6-8267-7ac418b2bdf0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:35.997303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556de4-a3fc-4c2b-ba87-472663320bde None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:35.997776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556de4-a3fc-4c2b-ba87-472663320bde None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:35.999396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 86/167] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:35 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:36.005655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19603b-8a1f-45df-90f4-300e1095c209 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.006252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19603b-8a1f-45df-90f4-300e1095c209 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.006858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19603b-8a1f-45df-90f4-300e1095c209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.035282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4d19603b-8a1f-45df-90f4-300e1095c209 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:36.037515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4d19603b-8a1f-45df-90f4-300e1095c209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:36.047338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:36.048410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.048919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.049403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.078067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:36.084317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:36.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:36.086392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:36.097833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 82/168] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:36.100492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:36.180926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:36.187180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 87/169] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:36.233444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19603b-8a1f-45df-90f4-300e1095c209 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:36.289614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac54b91-9c9a-4034-9ce7-3a3dba87c9ff None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:36.349551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:36.350660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.351188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.351593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.352486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:36.352925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:36.354478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:36.371840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 83/170] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:15:36 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:36.377800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:36.389532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:36.405353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19603b-8a1f-45df-90f4-300e1095c209 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:36.492890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bea2c-a522-4f0c-b122-487cedac8a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:36.494010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bea2c-a522-4f0c-b122-487cedac8a6a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.494434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bea2c-a522-4f0c-b122-487cedac8a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.494844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bea2c-a522-4f0c-b122-487cedac8a6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.507370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bea2c-a522-4f0c-b122-487cedac8a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b94bea2c-a522-4f0c-b122-487cedac8a6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:36.512615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:36.513701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:36.514690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:36.516636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 88/171] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:36.518086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4f40-60c9-491d-866d-01e3988bfe7b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.518516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4f40-60c9-491d-866d-01e3988bfe7b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.518949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4f40-60c9-491d-866d-01e3988bfe7b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.520718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 89/172] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:36.522530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf4f40-60c9-491d-866d-01e3988bfe7b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:36.553642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa6e92-8c47-4c63-901a-69ae8fe10d5a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:36.588374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:36.631546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa1b4f-04e8-41d2-83ca-16f42823e011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:36.632722 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa1b4f-04e8-41d2-83ca-16f42823e011 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.633219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa1b4f-04e8-41d2-83ca-16f42823e011 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.633686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa1b4f-04e8-41d2-83ca-16f42823e011 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.637264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa1b4f-04e8-41d2-83ca-16f42823e011 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:36.638815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa1b4f-04e8-41d2-83ca-16f42823e011 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:36.640654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 84/173] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:36 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:36.641843 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebab00c-b34b-4b5f-af2a-662850ef5d5d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.642112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebab00c-b34b-4b5f-af2a-662850ef5d5d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.642546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebab00c-b34b-4b5f-af2a-662850ef5d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.652989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf4f40-60c9-491d-866d-01e3988bfe7b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:36.680917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1ebab00c-b34b-4b5f-af2a-662850ef5d5d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:36.682153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1ebab00c-b34b-4b5f-af2a-662850ef5d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:36.689728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:36.690606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.691132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.692155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.693462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:36.693971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:36.695683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:36.705903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 90/174] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:36 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:36.706994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:36.794129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 85/175] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:36.795954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebab00c-b34b-4b5f-af2a-662850ef5d5d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:36.798637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:36.860913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934a13b-e3e7-4c68-864f-3e47da6edf5a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:36.889547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:36.904074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b85ac-beae-4672-8ba3-b4ac25e67630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:36.904727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b85ac-beae-4672-8ba3-b4ac25e67630 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.905010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b85ac-beae-4672-8ba3-b4ac25e67630 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.905283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b85ac-beae-4672-8ba3-b4ac25e67630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.925830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebab00c-b34b-4b5f-af2a-662850ef5d5d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:36.961212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec01c75-64eb-4017-8993-554f5cf0fcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:36.961859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01c75-64eb-4017-8993-554f5cf0fcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.962142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01c75-64eb-4017-8993-554f5cf0fcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.962416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01c75-64eb-4017-8993-554f5cf0fcd3 None admin] PATH_INFO: `/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.963629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec01c75-64eb-4017-8993-554f5cf0fcd3 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d7c3cdb6c4cc9a4f4f8447cb9227c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:36.965294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec01c75-64eb-4017-8993-554f5cf0fcd3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:36.966917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 86/176] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:36 2022] GET /identity/v3/users/af6d7c3cdb6c4cc9a4f4f8447cb9227c => generated 276 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:36.968049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676b306-293c-4bad-acfd-02b038b77e5f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:36.968339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676b306-293c-4bad-acfd-02b038b77e5f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:36.968652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676b306-293c-4bad-acfd-02b038b77e5f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:36.969866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 87/177] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:36 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:36.972624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676b306-293c-4bad-acfd-02b038b77e5f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:36.978869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b85ac-beae-4672-8ba3-b4ac25e67630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:36.980351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b85ac-beae-4672-8ba3-b4ac25e67630 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:37.042920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.075086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676b306-293c-4bad-acfd-02b038b77e5f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.080327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 91/178] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:36 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:37.080327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c5347-a3d5-477b-b861-7cfb320f7e88 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.080327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c5347-a3d5-477b-b861-7cfb320f7e88 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.080327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c5347-a3d5-477b-b861-7cfb320f7e88 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.080327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 92/179] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:37.080327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb1861-47d3-4e16-9a36-6a2943a11890 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.085668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb1861-47d3-4e16-9a36-6a2943a11890 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.085668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb1861-47d3-4e16-9a36-6a2943a11890 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.086978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 93/180] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:37.087775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb1861-47d3-4e16-9a36-6a2943a11890 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.109816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.114436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d9836-22d3-413e-bfbe-08c584ede120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.115224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d9836-22d3-413e-bfbe-08c584ede120 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.115651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d9836-22d3-413e-bfbe-08c584ede120 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.115989 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d9836-22d3-413e-bfbe-08c584ede120 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.127691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d9836-22d3-413e-bfbe-08c584ede120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-389d9836-22d3-413e-bfbe-08c584ede120 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:37.133328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:37.134212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:37.134996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 88/181] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:37.136152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdc847-a9fe-4b60-b4e6-f453f14742ca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.136152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdc847-a9fe-4b60-b4e6-f453f14742ca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.136386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdc847-a9fe-4b60-b4e6-f453f14742ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.154956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1dfdc847-a9fe-4b60-b4e6-f453f14742ca None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:37.156156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1dfdc847-a9fe-4b60-b4e6-f453f14742ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:37.194991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb1861-47d3-4e16-9a36-6a2943a11890 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.225027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 89/182] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:37.231797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facb80b-b57d-405a-a7b0-94297e3aa4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.232494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facb80b-b57d-405a-a7b0-94297e3aa4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.232836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facb80b-b57d-405a-a7b0-94297e3aa4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.233005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdc847-a9fe-4b60-b4e6-f453f14742ca None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.233100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facb80b-b57d-405a-a7b0-94297e3aa4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.292730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.306340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5facb80b-b57d-405a-a7b0-94297e3aa4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:37.308429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5facb80b-b57d-405a-a7b0-94297e3aa4b7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:37.328904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.334106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdc847-a9fe-4b60-b4e6-f453f14742ca None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.369413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fde6b-de74-4f1b-aa30-0a62697a1b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.370085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fde6b-de74-4f1b-aa30-0a62697a1b85 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.370659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fde6b-de74-4f1b-aa30-0a62697a1b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.370935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fde6b-de74-4f1b-aa30-0a62697a1b85 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.372087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fde6b-de74-4f1b-aa30-0a62697a1b85 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:37.373564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fde6b-de74-4f1b-aa30-0a62697a1b85 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:37.375218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 90/183] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:37.376318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c717-dccd-4e9c-beea-ce4437d0ffa8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.376631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c717-dccd-4e9c-beea-ce4437d0ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.376910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c717-dccd-4e9c-beea-ce4437d0ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.396339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b4b8c717-dccd-4e9c-beea-ce4437d0ffa8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:37.397567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b4b8c717-dccd-4e9c-beea-ce4437d0ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:37.408794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 94/184] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:37.410912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abda78-0d4c-4f0e-a599-4e946039a70b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.411484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abda78-0d4c-4f0e-a599-4e946039a70b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.411674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abda78-0d4c-4f0e-a599-4e946039a70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.429055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c8abda78-0d4c-4f0e-a599-4e946039a70b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:37.430247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c8abda78-0d4c-4f0e-a599-4e946039a70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:37.469746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 91/185] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:37.470919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c717-dccd-4e9c-beea-ce4437d0ffa8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.502677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 95/186] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:37.505499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abda78-0d4c-4f0e-a599-4e946039a70b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.531292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.563944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.573992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c717-dccd-4e9c-beea-ce4437d0ffa8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.609729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.610283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.610575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.610854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.611580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:37.611893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:37.613314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:37.614781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abda78-0d4c-4f0e-a599-4e946039a70b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.621718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 92/187] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:37.622517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.649281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.649918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.650194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.650468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.651606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:37.653084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:37.654654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 96/188] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/domains/default => generated 200 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:37.659990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.722614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055126cb-06ce-4f8f-ae10-9e8cf687d738 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.743870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.757934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.758564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.758844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.759127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.759917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] RBAC: Adding query filter params (role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:37.760231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:37.761774 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:37.762485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63fdc-d88a-4da9-9b57-830c6f41976f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.767468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 93/189] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/role_assignments?user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 610 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:37.768247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.795403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:37.815195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.815860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.816145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.816675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] PATH_INFO: `/v3/projects/c22837fa2021499b9127291e5d53069d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.817979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] RBAC: Authorizing `identity:get_project(project_id=c22837fa2021499b9127291e5d53069d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:37.819534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:37.821697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 97/190] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/projects/c22837fa2021499b9127291e5d53069d => generated 313 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:37.822419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.894406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853423fe-b024-4314-aa71-a38f13a0ad3b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.942385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.943464 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.943765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.944048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:37.946062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4afd6-c34f-4ada-a646-0102bb2b9724 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:37.957098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:37.962890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:37.963775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:37.964612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 94/191] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:37.969298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:37.986427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:37.987083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:37.987368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:37.987644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.000585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:38.006470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:38.007340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:38.008144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 98/192] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:38.008880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.032667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:38.060704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:38.087024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0877efd-8a19-4130-a67c-b6d43da255e5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.115231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1e945-4679-432a-9f46-ce064063691b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.120733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.121385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.121672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.121950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] PATH_INFO: `/v3/users/04a63ce70dd9424ea4ae43a40a274c06` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.135876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] RBAC: Authorizing `identity:get_user(user_id=04a63ce70dd9424ea4ae43a40a274c06)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.137398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.139257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 95/193] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:37 2022] GET /identity/v3/users/04a63ce70dd9424ea4ae43a40a274c06 => generated 284 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:38.139804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.154788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.155351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.155636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.155917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.156648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.158132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.240665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a71f-6661-4401-83f0-44ed1a50b908 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.242164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:38.266867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 99/194] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:15:38 2022] POST /identity/v3/users => generated 298 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:38.267786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.276184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.276876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.277162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.277503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.278273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=c22837fa2021499b9127291e5d53069d, user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:38.278598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.280113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.286903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 96/195] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/role_assignments?user.id=af6d7c3cdb6c4cc9a4f4f8447cb9227c&scope.project.id=c22837fa2021499b9127291e5d53069d&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 610 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:38.287776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.409761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40753ef-a5f4-41b9-b163-7730fff5e801 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.411982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:38.422264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453f6e4-fad3-4287-b499-daa04aca8946 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.459229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.462202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.462505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.462800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.463547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:38.463863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.466076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.470256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.471001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.471291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.471567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.472296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:38.472658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.474335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.477716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 100/196] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:38.479033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.491733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 97/197] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:38.492916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.530945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:38.626810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f887d-933d-416c-9d68-e6648c241ffc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.635143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45efa86-ca25-4964-81f5-318bbcae7d42 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.670108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.670655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.670940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.671220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.677606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.678276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.678564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.678847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] PATH_INFO: `/v3/users/04a63ce70dd9424ea4ae43a40a274c06` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.680095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] RBAC: Authorizing `identity:get_user(user_id=04a63ce70dd9424ea4ae43a40a274c06)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.682075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.683231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 98/198] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/users/04a63ce70dd9424ea4ae43a40a274c06 => generated 284 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:38.684014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.708334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:38.755167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.756932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.824438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:38.833256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b0e41-e788-43b2-b9d4-33254768d9cc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:38.874047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 101/199] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:38.875294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.886295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:38.888050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:38.888373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:38.888674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] PATH_INFO: `/v3/users/5fd2d30736e443fcbba05e30baa54e7c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:38.891418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=5fd2d30736e443fcbba05e30baa54e7c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:38.896260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:38.903313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 99/200] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/users/5fd2d30736e443fcbba05e30baa54e7c => generated 290 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:38.903313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:38.969337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:39.018655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ced5-6028-4908-a5eb-a3caf8915db0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:39.031942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc2a1f-d665-42a7-a21d-f4ede377dbb4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:39.055455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:39.056086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:39.056693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:39.056985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:39.077490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:39.078632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:39.079096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:39.079696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:39.081390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:39.083707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:39.086455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 100/201] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:39.087572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:39.129321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:39.148978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:39.150870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:39.222344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62148c-1642-4a5d-860d-88baf30001f5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:39.227112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:39.258632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:39.259403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:39.259774 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:39.260062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:39.315094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 102/202] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:38 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:39.316065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:39.373033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:39.374819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:39.406750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:39.423626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09896c89-9a3b-4aa5-bb03-d5eb5ce35393 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:39.465312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:39.466348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:39.466348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:39.466647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:39.467589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=04a63ce70dd9424ea4ae43a40a274c06) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:39.467974 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:39.469468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:39.476511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 103/203] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:39 2022] GET /identity/v3/role_assignments?user.id=04a63ce70dd9424ea4ae43a40a274c06&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:39.482664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:39.495717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 101/204] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:39.497329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:39.532439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:39.615805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dbe17-901d-43e6-8379-1345087c818d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:39.651836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:39.658356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:39.659377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:39.659900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:39.660648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:39.662385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:39.664165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:39.671120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 102/205] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:39 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:39.673869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:39.740114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fb53e-d39d-4367-8fe5-3163ad4f8b30 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:39.804655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:39.805662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:39.806178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:39.806674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:39.810415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:39.812113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:39.817983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 104/206] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:39 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:39.823706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:39.859227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd58bc9-a80c-4c4b-b3f5-691067f6f6cd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:39.942723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c111d-db47-4b57-b6df-8136479e540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:39.943350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c111d-db47-4b57-b6df-8136479e540e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:39.943620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c111d-db47-4b57-b6df-8136479e540e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:39.943893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c111d-db47-4b57-b6df-8136479e540e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:39.944723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c111d-db47-4b57-b6df-8136479e540e None admin] RBAC: Adding query filter params (role.id=a9b476e637814e0f8b3a8aa8e5ca9578, scope.project.id=fec0664d0d794aee9dc2952586baef2b, user.id=5fd2d30736e443fcbba05e30baa54e7c) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:39.945036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c111d-db47-4b57-b6df-8136479e540e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:39.946489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c111d-db47-4b57-b6df-8136479e540e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:39.961367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 103/207] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:39 2022] GET /identity/v3/role_assignments?user.id=5fd2d30736e443fcbba05e30baa54e7c&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=a9b476e637814e0f8b3a8aa8e5ca9578 => generated 610 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:39.964772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:39.971158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1a7d7-f0fc-4c76-a68a-e7055d972afb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:40.010588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258eece7-3133-4450-8c0e-613962baf61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:40.011569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eece7-3133-4450-8c0e-613962baf61f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:40.012078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eece7-3133-4450-8c0e-613962baf61f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:40.012609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eece7-3133-4450-8c0e-613962baf61f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:40.013761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eece7-3133-4450-8c0e-613962baf61f None admin] RBAC: Adding query filter params (role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=04a63ce70dd9424ea4ae43a40a274c06) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:40.014913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eece7-3133-4450-8c0e-613962baf61f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:40.017082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eece7-3133-4450-8c0e-613962baf61f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:40.024283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 105/208] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:39 2022] GET /identity/v3/role_assignments?user.id=04a63ce70dd9424ea4ae43a40a274c06&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:40.637143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda69f7-9806-4e93-a211-7b72e8f98b15 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:40.637836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda69f7-9806-4e93-a211-7b72e8f98b15 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:40.638326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda69f7-9806-4e93-a211-7b72e8f98b15 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:40.640177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 104/209] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:40 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:40.646127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a3c62-47d9-40be-896f-e6544adda0de None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:40.646719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a3c62-47d9-40be-896f-e6544adda0de None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:40.646825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c677d98-281d-4349-953c-f7aedb81067e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:40.647081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c677d98-281d-4349-953c-f7aedb81067e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:40.647362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c677d98-281d-4349-953c-f7aedb81067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:40.647732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a3c62-47d9-40be-896f-e6544adda0de None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:40.649330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 106/210] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:40 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:40.656583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab9423-8435-4f9b-a80d-80ea6f25c154 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:40.657133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab9423-8435-4f9b-a80d-80ea6f25c154 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:40.657621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab9423-8435-4f9b-a80d-80ea6f25c154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:40.666934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0c677d98-281d-4349-953c-f7aedb81067e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:40.668132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0c677d98-281d-4349-953c-f7aedb81067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:40.680301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bfab9423-8435-4f9b-a80d-80ea6f25c154 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:40.681547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bfab9423-8435-4f9b-a80d-80ea6f25c154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:40.744069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:40.753401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 105/211] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:40.770488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 107/212] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:40.777457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c677d98-281d-4349-953c-f7aedb81067e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:40.801771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab9423-8435-4f9b-a80d-80ea6f25c154 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:40.896681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c677d98-281d-4349-953c-f7aedb81067e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:40.901086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:40.911913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab9423-8435-4f9b-a80d-80ea6f25c154 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:40.935719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:40.936377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:40.936691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:40.936971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:40.949883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:40.952159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:40.953090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:40.953577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:40.954057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:40.954910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:40.955779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:40.957046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 106/213] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:40.958721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:40.961092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:40.961953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:40.967715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 108/214] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:40 2022] GET /identity/v3/domains/default => generated 200 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:40.969363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:41.022969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:41.073054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a42a9-0582-42a3-b4b3-26d46d143700 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:41.076869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af7bae-73b7-4080-84b3-8f5a9ff8484b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:41.111630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:41.112236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:41.112547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:41.112828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:41.114217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:41.114566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:41.115911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:41.116929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:41.117781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:41.118273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:41.118765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:41.119658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:41.121591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:41.124948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 107/215] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:40 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:41.130632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:41.218551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 109/216] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:15:40 2022] POST /identity/v3/users => generated 298 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:41.255199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542a8f0-345a-41e4-8344-9af1487bc05f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:41.278419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:41.301703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:41.302803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:41.303412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:41.304033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] PATH_INFO: `/v3/users/0797d2c8621a4b1dbaeb3a22244e14b2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:41.322046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] RBAC: Authorizing `identity:get_user(user_id=0797d2c8621a4b1dbaeb3a22244e14b2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:41.324159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:41.326311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 108/217] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:41 2022] GET /identity/v3/users/0797d2c8621a4b1dbaeb3a22244e14b2 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:41.333201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:41.351252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:41.451479 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0714f3-e62d-4141-be00-3c0a3abe0d98 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:41.488507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50df78-40b0-4b35-a664-876089ea97df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:41.489146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50df78-40b0-4b35-a664-876089ea97df None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:41.489432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50df78-40b0-4b35-a664-876089ea97df None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:41.489711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50df78-40b0-4b35-a664-876089ea97df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:41.568622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50df78-40b0-4b35-a664-876089ea97df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:41.570134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50df78-40b0-4b35-a664-876089ea97df None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:41.588147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:41.684087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 110/218] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:41 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:41.689966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:41.695059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c40a2-bc7e-4240-bcfa-be744ba3face None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:41.695352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c40a2-bc7e-4240-bcfa-be744ba3face None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:41.695815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c40a2-bc7e-4240-bcfa-be744ba3face None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:41.697189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 111/219] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:41.702270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6be96-0b10-4200-b3d4-719042aaa804 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:41.702558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6be96-0b10-4200-b3d4-719042aaa804 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:41.703221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6be96-0b10-4200-b3d4-719042aaa804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:41.722410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a4d6be96-0b10-4200-b3d4-719042aaa804 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:41.724167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a4d6be96-0b10-4200-b3d4-719042aaa804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:41.861817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1201aa1-d7ac-4933-a5f2-125baeb94477 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:41.871121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 112/220] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:41.897558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6be96-0b10-4200-b3d4-719042aaa804 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:41.918112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:41.918827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:41.919116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:41.919395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:41.921073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:41.922997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:41.925400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 109/221] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:41 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:41.932180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:41.959585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:42.003029 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:42.044888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6be96-0b10-4200-b3d4-719042aaa804 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:42.116700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:42.117296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:42.117554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:42.117807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:42.131635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086917-1ee0-431e-9e8e-f7edb52f6959 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:42.153392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:42.158720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:42.159607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:42.160646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:42.169266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 113/222] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:42.173294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:42.178183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b0a87-e62d-4618-a986-ed05b65e2a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:42.182211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0a87-e62d-4618-a986-ed05b65e2a0a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:42.182725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0a87-e62d-4618-a986-ed05b65e2a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:42.183218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0a87-e62d-4618-a986-ed05b65e2a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:42.184228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b0a87-e62d-4618-a986-ed05b65e2a0a None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=0797d2c8621a4b1dbaeb3a22244e14b2) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:42.184790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b0a87-e62d-4618-a986-ed05b65e2a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:42.187222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b0a87-e62d-4618-a986-ed05b65e2a0a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:42.200707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 110/223] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:41 2022] GET /identity/v3/role_assignments?user.id=0797d2c8621a4b1dbaeb3a22244e14b2&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 260 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:42.324794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a0720-dfc2-47f9-8820-65feddb48ab3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:42.325429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a0720-dfc2-47f9-8820-65feddb48ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:42.325945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a0720-dfc2-47f9-8820-65feddb48ab3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:42.327720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 111/224] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:42.333924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18aab-7ada-4d35-99ae-2c737b043102 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:42.334512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18aab-7ada-4d35-99ae-2c737b043102 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:42.334994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18aab-7ada-4d35-99ae-2c737b043102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:42.368717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6093af5-e583-4f47-a9f4-803f694481e1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:42.371241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4de18aab-7ada-4d35-99ae-2c737b043102 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:42.372532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4de18aab-7ada-4d35-99ae-2c737b043102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:42.375672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:42.442856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:42.443808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:42.444250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:42.452032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:42.452032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:42.452032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:42.452032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:42.462630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 114/225] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:42 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:42.463599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:42.492765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 112/226] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:42.527127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de18aab-7ada-4d35-99ae-2c737b043102 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:42.596983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:42.656586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac127d8-5723-46c1-823b-012f2554718e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:42.712480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de18aab-7ada-4d35-99ae-2c737b043102 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:42.725088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:42.725752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:42.726036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:42.726312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] PATH_INFO: `/v3/users/04a63ce70dd9424ea4ae43a40a274c06` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:42.727532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] RBAC: Authorizing `identity:get_user(user_id=04a63ce70dd9424ea4ae43a40a274c06)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:42.729035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:42.730824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 115/227] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:42 2022] GET /identity/v3/users/04a63ce70dd9424ea4ae43a40a274c06 => generated 284 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:42.737350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:42.774472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:42.775557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:42.776060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:42.776592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:42.791001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:42.796786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:42.797663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:42.798459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:42.801822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 113/228] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:42.807836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:42.844873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:42.930946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e994f5c-706f-40c8-b1ef-587a8b3ce831 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:42.999559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:43.000801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.001297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.002030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.003431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:43.005790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:43.008015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 116/229] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:42 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:43.013514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:43.025594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:43.039714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2979f-6da6-4117-886f-59fd5d2ac830 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:43.118259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:43.118841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.119191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.119554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.120650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:43.121026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:43.122486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:43.136716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9929c-f6e2-41d8-82bb-2412713bb96a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:43.139373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:43.147035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 114/230] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:15:42 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:43.152570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:43.178754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:43.180248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.181204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.181694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/04a63ce70dd9424ea4ae43a40a274c06/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.183997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353, user_id=04a63ce70dd9424ea4ae43a40a274c06, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:43.185848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:43.285524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd157d-8815-4d2c-949b-2b7c32c559a5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:43.322863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 117/231] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:43 2022] PUT /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/04a63ce70dd9424ea4ae43a40a274c06/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:43.329103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:43.329783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.330121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.330402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] PATH_INFO: `/v3/users/04a63ce70dd9424ea4ae43a40a274c06` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.333732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] RBAC: Authorizing `identity:get_user(user_id=04a63ce70dd9424ea4ae43a40a274c06)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:43.335409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:43.337589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 115/232] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:43 2022] GET /identity/v3/users/04a63ce70dd9424ea4ae43a40a274c06 => generated 284 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:43.343054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:43.506211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8afe0-5b47-47e7-88bc-82898b4924c7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:43.553334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff4bb0-8aad-4df6-a6c7-6e63abd63521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:43.554004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4bb0-8aad-4df6-a6c7-6e63abd63521 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.554295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4bb0-8aad-4df6-a6c7-6e63abd63521 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.554576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4bb0-8aad-4df6-a6c7-6e63abd63521 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.555758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff4bb0-8aad-4df6-a6c7-6e63abd63521 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:43.557257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff4bb0-8aad-4df6-a6c7-6e63abd63521 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:43.559253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 118/233] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:43 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:43.564582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:43.600112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:43.689159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a467-5ed5-4bec-92ba-d614cc35572e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:43.745570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e333-a371-44cd-8064-217471159511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:43.746182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e333-a371-44cd-8064-217471159511 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.746465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e333-a371-44cd-8064-217471159511 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.746743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e333-a371-44cd-8064-217471159511 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/04a63ce70dd9424ea4ae43a40a274c06/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.749077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93e333-a371-44cd-8064-217471159511 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353, user_id=04a63ce70dd9424ea4ae43a40a274c06, role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:43.750520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93e333-a371-44cd-8064-217471159511 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:43.795652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dfd36-27a2-4ba4-b862-3bd8442d1143 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.795961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dfd36-27a2-4ba4-b862-3bd8442d1143 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.796226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dfd36-27a2-4ba4-b862-3bd8442d1143 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.797988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 119/234] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:43.803697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d6feb-ebf3-4c52-a115-ae6605ac1360 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:43.803999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d6feb-ebf3-4c52-a115-ae6605ac1360 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:43.804284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d6feb-ebf3-4c52-a115-ae6605ac1360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:43.823031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6c2d6feb-ebf3-4c52-a115-ae6605ac1360 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:43.824344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6c2d6feb-ebf3-4c52-a115-ae6605ac1360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:43.851842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:43.856757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 116/235] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:43 2022] PUT /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/04a63ce70dd9424ea4ae43a40a274c06/roles/364a95d0748e48e9a538b4e6be293f91 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:43.903621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 120/236] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:43.928654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e333-a371-44cd-8064-217471159511 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:44.051451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:44.068178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e333-a371-44cd-8064-217471159511 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:44.105893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:44.106555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.106852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.107136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.120174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:44.125974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:44.126869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:44.128160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 117/237] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:44.133927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b6488-b5f7-4c3c-a50d-771a9f53ff55 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.134227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b6488-b5f7-4c3c-a50d-771a9f53ff55 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.134510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b6488-b5f7-4c3c-a50d-771a9f53ff55 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.136130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 121/238] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:44.137461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:44.144488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d54180-029d-4de4-ac97-adc790f49d94 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.144488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d54180-029d-4de4-ac97-adc790f49d94 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.144488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d54180-029d-4de4-ac97-adc790f49d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.165990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-76d54180-029d-4de4-ac97-adc790f49d94 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:44.167743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-76d54180-029d-4de4-ac97-adc790f49d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:44.317400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167445b-0d1c-444a-88e4-b8e4e64cba56 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:44.317607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 122/239] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:44.346199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d54180-029d-4de4-ac97-adc790f49d94 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:44.391214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:44.393598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.393896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.394195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.395515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:44.395881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:44.397681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:44.419529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 118/240] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:44 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:44.433951 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:44.461133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:44.499330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:44.521858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d54180-029d-4de4-ac97-adc790f49d94 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:44.590662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:44.591241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.591531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.591813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.593719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:44.595187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:44.597141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 123/241] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:44 2022] GET /identity/v3/domains/default => generated 200 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:44.603086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:44.632837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2168609-3a11-4f0e-a1dc-50bc99a9e19a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:44.669233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9633bd0-cb62-489c-836f-6c5bb4d42608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:44.669878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9633bd0-cb62-489c-836f-6c5bb4d42608 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.670198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9633bd0-cb62-489c-836f-6c5bb4d42608 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.670430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9633bd0-cb62-489c-836f-6c5bb4d42608 None admin] PATH_INFO: `/v3/users/08f50d734d404c4e88a8cff348704d9b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.684739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9633bd0-cb62-489c-836f-6c5bb4d42608 None admin] RBAC: Authorizing `identity:get_user(user_id=08f50d734d404c4e88a8cff348704d9b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:44.686206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9633bd0-cb62-489c-836f-6c5bb4d42608 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:44.688218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 119/242] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:44 2022] GET /identity/v3/users/08f50d734d404c4e88a8cff348704d9b => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:44.689507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88b9c6-055e-46e6-be27-4aea8c76c026 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.689800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88b9c6-055e-46e6-be27-4aea8c76c026 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.690079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88b9c6-055e-46e6-be27-4aea8c76c026 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.691884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 120/243] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:44.697678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88b9c6-055e-46e6-be27-4aea8c76c026 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:44.752280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869677aa-43d1-4e76-8ae8-b00e843e61da None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:44.772105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:44.798207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fa2c8-7d19-48dd-9d59-d2be2c61ae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:44.799034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa2c8-7d19-48dd-9d59-d2be2c61ae40 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.799410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa2c8-7d19-48dd-9d59-d2be2c61ae40 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.799829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa2c8-7d19-48dd-9d59-d2be2c61ae40 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.800705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fa2c8-7d19-48dd-9d59-d2be2c61ae40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:44.802538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fa2c8-7d19-48dd-9d59-d2be2c61ae40 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:44.822438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88b9c6-055e-46e6-be27-4aea8c76c026 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:44.859642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:44.860298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.860635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.860914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.881172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 124/244] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:15:44 2022] POST /identity/v3/users => generated 294 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:44.883456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6fe27-11d3-46b3-9bbf-ea3fd18b29c1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:44.883987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6fe27-11d3-46b3-9bbf-ea3fd18b29c1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:44.884513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6fe27-11d3-46b3-9bbf-ea3fd18b29c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:44.911725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5df6fe27-11d3-46b3-9bbf-ea3fd18b29c1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:44.913744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5df6fe27-11d3-46b3-9bbf-ea3fd18b29c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:44.927532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:44.945110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:44.946831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:45.023667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 125/245] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:45.025283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:45.046765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6fe27-11d3-46b3-9bbf-ea3fd18b29c1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:45.069590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 121/246] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:45.076916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:45.190150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a630a3-d066-4f19-be61-fb680becdcea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:45.213655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6fe27-11d3-46b3-9bbf-ea3fd18b29c1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:45.229574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:45.230592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.231105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.231587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.233088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:45.234867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:45.237214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 122/247] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:45 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:45.244492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:45.258688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:45.259392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.262823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.262823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.272120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:45.277311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:45.278184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:45.279031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 126/248] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:45.284675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:45.301344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:45.330287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:45.414569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2261b18-ad4f-4be5-8598-0b02581fa7e2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:45.414759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a901-3dd1-4922-a249-386fb55b30da None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:45.457060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:45.457406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.457717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.457965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.458776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] RBAC: Adding query filter params (role.id=a9b476e637814e0f8b3a8aa8e5ca9578, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=08f50d734d404c4e88a8cff348704d9b) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:45.459098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:45.460885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:45.463833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:45.464954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.465245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.465524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.467218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:45.467218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:45.468247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:45.474990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 123/249] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:45 2022] GET /identity/v3/role_assignments?user.id=08f50d734d404c4e88a8cff348704d9b&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=a9b476e637814e0f8b3a8aa8e5ca9578 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:45.483628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 127/250] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:45 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:45.489765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:45.604015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b911837-250b-4e7b-b5fe-644a7fc07b43 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:45.606657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:45.643184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:45.643862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.644212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.645091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] PATH_INFO: `/v3/users/0797d2c8621a4b1dbaeb3a22244e14b2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.646530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] RBAC: Authorizing `identity:get_user(user_id=0797d2c8621a4b1dbaeb3a22244e14b2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:45.648543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:45.652839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 124/251] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:45 2022] GET /identity/v3/users/0797d2c8621a4b1dbaeb3a22244e14b2 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:45.656755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:45.730184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:45.768185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf6d39-fa64-4df6-88b7-72f6b671e388 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:45.804731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53593e-a8a8-4ad5-8424-f5265c3daab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:45.805502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53593e-a8a8-4ad5-8424-f5265c3daab6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.805785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53593e-a8a8-4ad5-8424-f5265c3daab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.806064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53593e-a8a8-4ad5-8424-f5265c3daab6 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.807256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53593e-a8a8-4ad5-8424-f5265c3daab6 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:45.808748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53593e-a8a8-4ad5-8424-f5265c3daab6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:45.810733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 128/252] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:45 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:45.817120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:45.911088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fea2f9-d1e5-43e0-80d1-be7e0641998c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.911407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fea2f9-d1e5-43e0-80d1-be7e0641998c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.911657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fea2f9-d1e5-43e0-80d1-be7e0641998c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.913046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 129/253] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:45.919331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19472edf-80a3-4c42-bf51-5e5764e82215 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.919635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19472edf-80a3-4c42-bf51-5e5764e82215 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.919900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19472edf-80a3-4c42-bf51-5e5764e82215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.931361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224b9b0-62e0-4685-b76a-74affdba42f3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:45.944907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-19472edf-80a3-4c42-bf51-5e5764e82215 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:45.946407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-19472edf-80a3-4c42-bf51-5e5764e82215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:45.972795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d42e81-27f7-4980-ab3b-ce58344fe6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:45.973857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d42e81-27f7-4980-ab3b-ce58344fe6a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:45.974385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d42e81-27f7-4980-ab3b-ce58344fe6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:45.974885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d42e81-27f7-4980-ab3b-ce58344fe6a5 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/0797d2c8621a4b1dbaeb3a22244e14b2/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:45.977334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d42e81-27f7-4980-ab3b-ce58344fe6a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353, user_id=0797d2c8621a4b1dbaeb3a22244e14b2, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:45.979035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d42e81-27f7-4980-ab3b-ce58344fe6a5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:46.079178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 130/254] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:46.108836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 125/255] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:45 2022] PUT /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/0797d2c8621a4b1dbaeb3a22244e14b2/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:46.123451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19472edf-80a3-4c42-bf51-5e5764e82215 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:46.206266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:46.250748 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284934f-f2e8-4666-90b9-578fa5579c37 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:46.251076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284934f-f2e8-4666-90b9-578fa5579c37 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:46.251305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284934f-f2e8-4666-90b9-578fa5579c37 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:46.253048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 126/256] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:46.259580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab182d7-2ed5-4c44-af5e-2f0b102acdbb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:46.259859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab182d7-2ed5-4c44-af5e-2f0b102acdbb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:46.260137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab182d7-2ed5-4c44-af5e-2f0b102acdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:46.263244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19472edf-80a3-4c42-bf51-5e5764e82215 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:46.323401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fab182d7-2ed5-4c44-af5e-2f0b102acdbb None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:46.324679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fab182d7-2ed5-4c44-af5e-2f0b102acdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:46.363199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:46.375093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:46.375755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:46.384503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:46.384503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:46.395949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:46.401040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:46.402018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:15:46.402791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:46.411287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 131/257] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:46.418444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:46.460100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 127/258] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:46.485258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab182d7-2ed5-4c44-af5e-2f0b102acdbb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:46.636638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab182d7-2ed5-4c44-af5e-2f0b102acdbb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:46.678621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:46.679280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:46.679561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:46.679836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:46.692896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:46.695165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f07f48-cd2a-4a8b-937d-7b8ac823de41 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:46.698335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:46.699198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:46.699975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 128/259] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:46.707080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:46.731448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:46.739060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:46.755166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:46.755802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:46.756085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:46.756360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:46.757135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:46.757446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:46.758945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:46.776245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 132/260] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:46 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:46.783106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:46.826698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a816c7-4239-46da-9a62-2cc82cc24a31 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:46.864944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:46.865811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:46.866235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:46.866755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:46.867616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:46.868061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:46.869861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:46.879684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 129/261] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:15:46 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:46.885289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:47.020776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:47.033714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f8ff6-bc64-413f-95e2-e70dbc258989 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:47.057330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c742-739b-4de7-b90e-c976704fe2a4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:47.114694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:47.116518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:47.116947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:47.117347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:47.117599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:47.117882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:47.118153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] PATH_INFO: `/v3/users/04a63ce70dd9424ea4ae43a40a274c06` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:47.118552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] PATH_INFO: `/v3/users/04a63ce70dd9424ea4ae43a40a274c06` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:47.120495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] RBAC: Authorizing `identity:get_user(user_id=04a63ce70dd9424ea4ae43a40a274c06)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:47.122257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] RBAC: Authorizing `identity:get_user(user_id=04a63ce70dd9424ea4ae43a40a274c06)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:47.122589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:47.128658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:47.128658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 133/262] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:46 2022] GET /identity/v3/users/04a63ce70dd9424ea4ae43a40a274c06 => generated 284 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:47.128954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 130/263] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:46 2022] GET /identity/v3/users/04a63ce70dd9424ea4ae43a40a274c06 => generated 284 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:47.131622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:47.132356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:47.255258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:47.348940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c084c66-bec8-4edb-ac07-72cf67f9c374 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:47.358616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc56f1-10cf-4d86-bb7e-b7240988866b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:47.440068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:47.440725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:47.441197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:47.441197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:47.458022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:47.458579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:47.458872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:47.459155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:47.498675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:47.596878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:47.600265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:47.626514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:47.627993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:47.716813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:47.819921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 134/264] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:47.825525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:47.843650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 131/265] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:15:47.850254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:47.931609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:48.033818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813f4e0-bd64-44b6-8d89-7198af1a90bd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:48.070854 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c6189-a0ff-4995-8cc9-b81500699a92 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:48.106168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:48.107287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.107906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.108378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.112555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:48.115123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:48.118640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 132/266] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:47 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:48.123554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:48.139426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:48.139426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.139426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.139426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.139817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:48.141153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:48.142958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 135/267] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:47 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:48.151877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:48.208434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:48.314822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f4c10-8e05-44a6-a0ad-2d4e71f34484 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:48.344152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:48.384714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156a57c-31fc-4ab0-aca6-3ec64022a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:48.385759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a57c-31fc-4ab0-aca6-3ec64022a684 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.386428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a57c-31fc-4ab0-aca6-3ec64022a684 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.386898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a57c-31fc-4ab0-aca6-3ec64022a684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.387918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156a57c-31fc-4ab0-aca6-3ec64022a684 None admin] RBAC: Adding query filter params (role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=04a63ce70dd9424ea4ae43a40a274c06) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:48.388450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156a57c-31fc-4ab0-aca6-3ec64022a684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:48.390109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156a57c-31fc-4ab0-aca6-3ec64022a684 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:48.391406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd8f07-66f6-4661-adc6-b0a425249ca0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:48.406132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 133/268] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:48 2022] GET /identity/v3/role_assignments?user.id=04a63ce70dd9424ea4ae43a40a274c06&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 610 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:48.407861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5cf79-1b17-4722-81ad-af87fbc29244 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.408344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5cf79-1b17-4722-81ad-af87fbc29244 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.408846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5cf79-1b17-4722-81ad-af87fbc29244 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.411393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 134/269] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:48.418077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0cc62-9eed-4a04-992b-77d3ae1cd585 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.418769 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0cc62-9eed-4a04-992b-77d3ae1cd585 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.419262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0cc62-9eed-4a04-992b-77d3ae1cd585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.451474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-23e0cc62-9eed-4a04-992b-77d3ae1cd585 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:48.451474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-23e0cc62-9eed-4a04-992b-77d3ae1cd585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:48.457772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:48.457772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.457772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.457772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.458172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=04a63ce70dd9424ea4ae43a40a274c06) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:48.458172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:48.458925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:48.479105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 136/270] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:48 2022] GET /identity/v3/role_assignments?user.id=04a63ce70dd9424ea4ae43a40a274c06&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:48.600190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 135/271] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:48.640721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:48.668156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03fec2-3199-40b9-bab9-149fc4ed72da None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.668515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03fec2-3199-40b9-bab9-149fc4ed72da None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.668806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03fec2-3199-40b9-bab9-149fc4ed72da None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.670517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 136/272] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:48.678191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a243bb-f29d-4390-94cf-7d7d9886d186 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.678660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a243bb-f29d-4390-94cf-7d7d9886d186 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.678976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a243bb-f29d-4390-94cf-7d7d9886d186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.710679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-47a243bb-f29d-4390-94cf-7d7d9886d186 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:48.711920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-47a243bb-f29d-4390-94cf-7d7d9886d186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:48.805574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:48.834574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5c53-fbeb-43d7-9a9b-a5a510136d27 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:48.889810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 137/273] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:48.891842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5ace-de27-4f60-a70b-69a110abc61d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.892333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5ace-de27-4f60-a70b-69a110abc61d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.892841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5ace-de27-4f60-a70b-69a110abc61d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.894787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 138/274] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:48.905276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79588-99fd-44a1-97d8-c5ae3b4ccc5c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.907432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79588-99fd-44a1-97d8-c5ae3b4ccc5c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.907927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79588-99fd-44a1-97d8-c5ae3b4ccc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.910138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:48.910803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:48.911090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:48.911370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:48.928024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:48.935207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:48.936446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:48.937888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:48.940508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 137/275] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:48.941643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:48.942967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c5b79588-99fd-44a1-97d8-c5ae3b4ccc5c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:48.945373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c5b79588-99fd-44a1-97d8-c5ae3b4ccc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:48.984072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.029027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 139/276] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:49.029886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b79588-99fd-44a1-97d8-c5ae3b4ccc5c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.079326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8321d-84cc-4883-8753-16496e82d23b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.125833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.126488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.126771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.127049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.139632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:49.146477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:49.147617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:49.148401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 138/277] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:49.153799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.153952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b79588-99fd-44a1-97d8-c5ae3b4ccc5c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.169252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.200869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.201622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.201945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.202771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.203506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:49.203802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.205227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.221661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 140/278] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:49.223757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.271216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.290337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5bef3-6ec3-44fd-9a63-5329cae65cda None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.334370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.335315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.335599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.335877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.337225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.339118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.340766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 139/279] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/domains/default => generated 200 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:49.341542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.358974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595a0d-4fe5-4635-a2dd-04e7805e16da None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.398270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173db8d7-0603-4864-94b4-694d4b253a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.398905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173db8d7-0603-4864-94b4-694d4b253a3b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.399191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173db8d7-0603-4864-94b4-694d4b253a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.399470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173db8d7-0603-4864-94b4-694d4b253a3b None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.400199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173db8d7-0603-4864-94b4-694d4b253a3b None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:49.400544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173db8d7-0603-4864-94b4-694d4b253a3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.401967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173db8d7-0603-4864-94b4-694d4b253a3b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.411364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 141/280] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:49.412642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f887ac-00aa-48d8-8ea8-1b6e7ffff804 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.412939 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f887ac-00aa-48d8-8ea8-1b6e7ffff804 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.413222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f887ac-00aa-48d8-8ea8-1b6e7ffff804 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.418592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 142/281] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:49.418592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f887ac-00aa-48d8-8ea8-1b6e7ffff804 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.440278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.471468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4084-c131-498b-9c7d-ba4b35f97ca6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.506595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.507235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.507512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.507784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] PATH_INFO: `/v3/users/08f50d734d404c4e88a8cff348704d9b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.508956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=08f50d734d404c4e88a8cff348704d9b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.510423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.512014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 140/282] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/users/08f50d734d404c4e88a8cff348704d9b => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:49.512790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.536548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.546129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f887ac-00aa-48d8-8ea8-1b6e7ffff804 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.581886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.582534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.582938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.583217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.583910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.585563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.625733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783025a-1061-4c7e-8b51-e34665125bd8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.658574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7303d-d851-4e34-8ef1-c23df55083c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.659253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7303d-d851-4e34-8ef1-c23df55083c2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.660249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7303d-d851-4e34-8ef1-c23df55083c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.660558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7303d-d851-4e34-8ef1-c23df55083c2 None admin] PATH_INFO: `/v3/users/0876821669e74bed8463e9ae4a32ec9f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.671651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.674090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7303d-d851-4e34-8ef1-c23df55083c2 None admin] RBAC: Authorizing `identity:get_user(user_id=0876821669e74bed8463e9ae4a32ec9f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.675560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7303d-d851-4e34-8ef1-c23df55083c2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.677222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 141/283] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/users/0876821669e74bed8463e9ae4a32ec9f => generated 294 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:49.678360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342d2c-fc7a-43af-9edd-64286472ab2f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.678703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342d2c-fc7a-43af-9edd-64286472ab2f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.678985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342d2c-fc7a-43af-9edd-64286472ab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.694574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 143/284] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:15:49 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:49.695406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.696753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4f342d2c-fc7a-43af-9edd-64286472ab2f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:49.698088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4f342d2c-fc7a-43af-9edd-64286472ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:49.780168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 142/285] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:49.781385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f342d2c-fc7a-43af-9edd-64286472ab2f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.835672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5a36c-1695-484a-879e-24b0e5fb120d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.876131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.876868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.877154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.877434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.880174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.881818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.883525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 144/286] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:49.885217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:49.908796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.920617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f342d2c-fc7a-43af-9edd-64286472ab2f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:49.953544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:49.964377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:49.965203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:49.965497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:49.965784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:49.966596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, user.id=0876821669e74bed8463e9ae4a32ec9f, scope.system=all) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:49.967018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:49.968565 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:49.976219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 143/287] 10.222.0.128 () {56 vars in 1313 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/role_assignments?user.id=0876821669e74bed8463e9ae4a32ec9f&scope.system=all&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 227 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:49.978249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:50.029084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5e4f5-5291-4f45-9a24-ba0609f20cfb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:50.078431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:50.080020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:50.080313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:50.080666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:50.097131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:50.105448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:50.106398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:50.107269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 145/288] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:50.111193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb8ef0-f9a4-4fe3-ae9b-612ac1ef9d01 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:50.113179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:50.163815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:50.164527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:50.164815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:50.165093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/08f50d734d404c4e88a8cff348704d9b/roles/a9b476e637814e0f8b3a8aa8e5ca9578` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:50.167540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353, user_id=08f50d734d404c4e88a8cff348704d9b, role_id=a9b476e637814e0f8b3a8aa8e5ca9578)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:50.169060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:50.218775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:50.240041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 144/289] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:15:49 2022] PUT /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/users/08f50d734d404c4e88a8cff348704d9b/roles/a9b476e637814e0f8b3a8aa8e5ca9578 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:50.245061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967927c0-18e0-45c5-917f-5cc07f84a267 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:50.284101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:50.285487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:50.285991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:50.286464 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:50.287688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:50.288007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:50.289490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:50.299812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 146/290] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:50 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:50.306229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:50.377172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:50.447622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213a3b-2dd0-4741-bd62-c8f8bfa98494 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:50.500896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:50.501559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:50.501846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:50.502126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] PATH_INFO: `/v3/users/0797d2c8621a4b1dbaeb3a22244e14b2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:50.505138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] RBAC: Authorizing `identity:get_user(user_id=0797d2c8621a4b1dbaeb3a22244e14b2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:50.506723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:50.508787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 145/291] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:50 2022] GET /identity/v3/users/0797d2c8621a4b1dbaeb3a22244e14b2 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:50.514368 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:50.639242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c6a10-c62f-4385-95eb-3bff664f0100 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:50.679173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:50.680177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:50.680731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:50.681219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:50.690792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:50.754782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:50.756269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:50.861414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 147/292] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:50 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:50.867347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:51.019379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2126edd-95cf-4aea-ad22-ff3330130c50 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:51.046018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:51.057800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a66d-21fd-4371-a995-4ff2f4faecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:51.058455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a66d-21fd-4371-a995-4ff2f4faecf0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:51.067802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a66d-21fd-4371-a995-4ff2f4faecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:51.067802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a66d-21fd-4371-a995-4ff2f4faecf0 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:51.067802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929a66d-21fd-4371-a995-4ff2f4faecf0 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:51.067802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929a66d-21fd-4371-a995-4ff2f4faecf0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:51.067802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 146/293] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:50 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:51.069423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:51.112064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:51.202433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06219923-4a71-4e4f-adbf-f27ec91501d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:51.253579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e41c2-77b4-4b76-a0a1-83355dbd2a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:51.254437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e41c2-77b4-4b76-a0a1-83355dbd2a4e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:51.254723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e41c2-77b4-4b76-a0a1-83355dbd2a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:51.255002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e41c2-77b4-4b76-a0a1-83355dbd2a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:51.255800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e41c2-77b4-4b76-a0a1-83355dbd2a4e None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=0797d2c8621a4b1dbaeb3a22244e14b2) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:51.256166 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e41c2-77b4-4b76-a0a1-83355dbd2a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:51.257632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e41c2-77b4-4b76-a0a1-83355dbd2a4e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:51.271531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 148/294] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:51 2022] GET /identity/v3/role_assignments?user.id=0797d2c8621a4b1dbaeb3a22244e14b2&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 610 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:52.242704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0cdc5-770e-4622-a9b3-370a96f3871c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.243451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0cdc5-770e-4622-a9b3-370a96f3871c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.243451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0cdc5-770e-4622-a9b3-370a96f3871c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.245192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 147/295] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:52.250526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede7c18-8e87-47ae-a44f-81d92cdbd092 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.253475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede7c18-8e87-47ae-a44f-81d92cdbd092 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.253475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede7c18-8e87-47ae-a44f-81d92cdbd092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.271600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4ede7c18-8e87-47ae-a44f-81d92cdbd092 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:52.273159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4ede7c18-8e87-47ae-a44f-81d92cdbd092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:52.290125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa19cd-2911-4fca-ba7d-e551c50a9549 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.290738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa19cd-2911-4fca-ba7d-e551c50a9549 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.291213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa19cd-2911-4fca-ba7d-e551c50a9549 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.293050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 148/296] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:52.298275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee4c72-9512-4541-9268-f5bc3e6e4b26 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.298792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee4c72-9512-4541-9268-f5bc3e6e4b26 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.299266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee4c72-9512-4541-9268-f5bc3e6e4b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.319833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-46ee4c72-9512-4541-9268-f5bc3e6e4b26 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:52.321400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-46ee4c72-9512-4541-9268-f5bc3e6e4b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:52.350712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 149/297] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:52.373480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede7c18-8e87-47ae-a44f-81d92cdbd092 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:52.400243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 149/298] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:52.408502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:52.424248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee4c72-9512-4541-9268-f5bc3e6e4b26 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:52.494679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede7c18-8e87-47ae-a44f-81d92cdbd092 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:52.532515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:52.533446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.533914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.535022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.536405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:52.538139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:52.544562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 150/299] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:52 2022] GET /identity/v3/domains/default => generated 200 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:52.545728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:52.586615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee4c72-9512-4541-9268-f5bc3e6e4b26 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:52.621508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:52.635881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196029d-0e38-4d29-ba46-42bb043c6dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:52.636634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196029d-0e38-4d29-ba46-42bb043c6dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.636913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196029d-0e38-4d29-ba46-42bb043c6dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.637182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196029d-0e38-4d29-ba46-42bb043c6dc6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.648265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196029d-0e38-4d29-ba46-42bb043c6dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6196029d-0e38-4d29-ba46-42bb043c6dc6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:52.653602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:52.654508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:52.655288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 150/300] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:52.656546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ed0dd-c7df-411f-b27f-c03af252ee43 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.656820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ed0dd-c7df-411f-b27f-c03af252ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.657096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ed0dd-c7df-411f-b27f-c03af252ee43 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.658972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 151/301] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:52.661291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ed0dd-c7df-411f-b27f-c03af252ee43 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:52.664574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:52.667833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372afd2-7059-4845-ac97-e8912f1f32b6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:52.701965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fdcf6-cd20-4706-b651-6c04d7bf9a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:52.702588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdcf6-cd20-4706-b651-6c04d7bf9a44 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.702867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdcf6-cd20-4706-b651-6c04d7bf9a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.703140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdcf6-cd20-4706-b651-6c04d7bf9a44 None admin] PATH_INFO: `/v3/groups` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.703823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fdcf6-cd20-4706-b651-6c04d7bf9a44 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:52.705283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fdcf6-cd20-4706-b651-6c04d7bf9a44 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:52.773297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ed0dd-c7df-411f-b27f-c03af252ee43 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:52.778175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 151/302] 10.222.0.128 () {60 vars in 1139 bytes} [Tue Jul 26 15:15:52 2022] POST /identity/v3/groups => generated 230 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:52.778175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c63f-b867-4439-b050-3bc05d299960 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.778175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c63f-b867-4439-b050-3bc05d299960 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.778175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c63f-b867-4439-b050-3bc05d299960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.805840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8e49c63f-b867-4439-b050-3bc05d299960 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:52.807545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e49c63f-b867-4439-b050-3bc05d299960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:52.819062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7582-7989-4849-9b8d-01a0f76e9423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:52.819694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7582-7989-4849-9b8d-01a0f76e9423 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.819979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7582-7989-4849-9b8d-01a0f76e9423 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.820285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7582-7989-4849-9b8d-01a0f76e9423 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.821030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d7582-7989-4849-9b8d-01a0f76e9423 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:52.821361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d7582-7989-4849-9b8d-01a0f76e9423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:52.822793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d7582-7989-4849-9b8d-01a0f76e9423 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:52.837607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 152/303] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:52 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:52.837607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e1f70-47b7-4eb2-bf24-668eda6fe3fa None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.837607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e1f70-47b7-4eb2-bf24-668eda6fe3fa None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.837607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e1f70-47b7-4eb2-bf24-668eda6fe3fa None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.837607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 153/304] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:52.841639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e1f70-47b7-4eb2-bf24-668eda6fe3fa None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:52.896543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 152/305] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:52.896798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:52.896887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9785ce-c8e1-4b5d-9c7b-df5e7ee04999 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:52.896887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9785ce-c8e1-4b5d-9c7b-df5e7ee04999 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:52.896887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9785ce-c8e1-4b5d-9c7b-df5e7ee04999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:52.918730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4d9785ce-c8e1-4b5d-9c7b-df5e7ee04999 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:52.919954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4d9785ce-c8e1-4b5d-9c7b-df5e7ee04999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:52.984062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e1f70-47b7-4eb2-bf24-668eda6fe3fa None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.004051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 153/306] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:53.004948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9785ce-c8e1-4b5d-9c7b-df5e7ee04999 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.022626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.023609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.024193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.024721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] PATH_INFO: `/v3/users/37814491ff224d2e95f79a5147d44c44` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.032238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:53.039982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] RBAC: Authorizing `identity:get_user(user_id=37814491ff224d2e95f79a5147d44c44)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:53.041490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:53.043395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 154/307] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:52 2022] GET /identity/v3/users/37814491ff224d2e95f79a5147d44c44 => generated 294 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:53.044122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.119058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9785ce-c8e1-4b5d-9c7b-df5e7ee04999 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.151130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:53.154345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5f867-ff1a-41cb-a337-4fb3544d09fb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.156451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.157129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.157408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.157681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.169207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:53.174126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:53.174978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 154/308] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:53.180913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.197876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.198609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.198900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.199297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.210633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:53.215688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:53.217725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:53.219579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:53.222404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 155/309] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:53.223342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.280349 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:53.304154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fcf92-2d76-4838-a599-3aec4b72d2f9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.338287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff496d1-3ed9-41e8-a6ce-86b26e67af0e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.340043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.340669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.340955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.341230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.342030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] RBAC: Adding query filter params (role.id=a9b476e637814e0f8b3a8aa8e5ca9578, user.id=37814491ff224d2e95f79a5147d44c44, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:53.342352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:53.343811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:53.359806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 155/310] 10.222.0.128 () {56 vars in 1313 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/role_assignments?user.id=37814491ff224d2e95f79a5147d44c44&scope.system=all&role.id=a9b476e637814e0f8b3a8aa8e5ca9578 => generated 227 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:53.359806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.380454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.381880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.381880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.381880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.382502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:53.382812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:53.384255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:53.399577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 156/311] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:53.400586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.430538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:53.527743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefae65-387e-427b-8d7c-4d7221989918 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.527937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ce230-6c20-417a-8cab-2d7f127f2f2b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.569112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.569667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.569955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.570233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.570744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.570970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:53.571284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:53.571943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.572408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.572694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:53.573069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] PATH_INFO: `/v3/users/0876821669e74bed8463e9ae4a32ec9f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.574791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] RBAC: Authorizing `identity:get_user(user_id=0876821669e74bed8463e9ae4a32ec9f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:53.576957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:53.579881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 157/312] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/users/0876821669e74bed8463e9ae4a32ec9f => generated 294 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:53.588557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 156/313] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:53.588989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.589468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.599841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:53.703008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31327cd-3eb4-45e3-ad09-d26798dc7623 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.713954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864c763-ecb8-4ad4-9f48-a46c24a319a8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.716830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:53.749208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.750028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.750159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.750423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.750699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.750809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] PATH_INFO: `/v3/users/08f50d734d404c4e88a8cff348704d9b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.750988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.751304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] PATH_INFO: `/v3/system/users/0876821669e74bed8463e9ae4a32ec9f/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:53.752299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] RBAC: Authorizing `identity:get_user(user_id=08f50d734d404c4e88a8cff348704d9b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:53.752848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0876821669e74bed8463e9ae4a32ec9f, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:53.753937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:53.754695 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:53.755843 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 157/314] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/users/08f50d734d404c4e88a8cff348704d9b => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:53.761631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:53.845012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:53.860966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 158/315] 10.222.0.128 () {58 vars in 1339 bytes} [Tue Jul 26 15:15:53 2022] PUT /identity/v3/system/users/0876821669e74bed8463e9ae4a32ec9f/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:53.880996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5535c-1958-498a-89c7-22b8128ca689 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:53.921839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:53.923341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:53.924258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:53.924962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:54.006087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:54.007970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:54.091874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:54.118953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 158/316] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:53 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:54.124787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:54.232773 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf59e-58ba-4976-8f57-ea2601e71ea2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:54.270130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6db3d7-7ac0-4214-be4e-99577f62178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:54.270790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6db3d7-7ac0-4214-be4e-99577f62178e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:54.271080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6db3d7-7ac0-4214-be4e-99577f62178e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:54.271362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6db3d7-7ac0-4214-be4e-99577f62178e None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:54.272574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6db3d7-7ac0-4214-be4e-99577f62178e None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:54.274048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6db3d7-7ac0-4214-be4e-99577f62178e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:54.276029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 159/317] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:54 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:54.282077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:54.395336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7fe82-f051-4c03-9968-71f3eec0ab12 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:54.432207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2264b89-be49-4f94-a2a8-529f5e34dd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:54.433252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264b89-be49-4f94-a2a8-529f5e34dd3f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:54.433752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264b89-be49-4f94-a2a8-529f5e34dd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:54.434227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264b89-be49-4f94-a2a8-529f5e34dd3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:54.435220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2264b89-be49-4f94-a2a8-529f5e34dd3f None admin] RBAC: Adding query filter params (role.id=a9b476e637814e0f8b3a8aa8e5ca9578, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353, user.id=08f50d734d404c4e88a8cff348704d9b) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:54.435746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2264b89-be49-4f94-a2a8-529f5e34dd3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:54.437421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2264b89-be49-4f94-a2a8-529f5e34dd3f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:54.444888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 159/318] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:15:54 2022] GET /identity/v3/role_assignments?user.id=08f50d734d404c4e88a8cff348704d9b&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=a9b476e637814e0f8b3a8aa8e5ca9578 => generated 610 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:54.935087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29de22-be95-4b25-b80a-de40f8bd4db4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:54.935628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29de22-be95-4b25-b80a-de40f8bd4db4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:54.935628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29de22-be95-4b25-b80a-de40f8bd4db4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:54.937034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 160/319] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:54.942332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f8547-2c00-4b38-9155-cad443551a41 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:54.942709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f8547-2c00-4b38-9155-cad443551a41 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:54.942999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f8547-2c00-4b38-9155-cad443551a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:54.962363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9d8f8547-2c00-4b38-9155-cad443551a41 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:54.963571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9d8f8547-2c00-4b38-9155-cad443551a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:54.996143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:55.034827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 160/320] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:55.057559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29de22-be95-4b25-b80a-de40f8bd4db4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:55.085118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:55.176115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29de22-be95-4b25-b80a-de40f8bd4db4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:55.210935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd9a9c-3915-4e49-ad53-eedde48eb178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:55.211592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd9a9c-3915-4e49-ad53-eedde48eb178 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:55.211877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd9a9c-3915-4e49-ad53-eedde48eb178 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:55.212154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd9a9c-3915-4e49-ad53-eedde48eb178 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:55.213333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd9a9c-3915-4e49-ad53-eedde48eb178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:55.214784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd9a9c-3915-4e49-ad53-eedde48eb178 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:55.216553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 161/321] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:15:55 2022] GET /identity/v3/domains/default => generated 200 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:55.222634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f8547-2c00-4b38-9155-cad443551a41 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:55.352014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f8547-2c00-4b38-9155-cad443551a41 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:55.389776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:55.390407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:55.390692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:55.390970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] PATH_INFO: `/v3/groups` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:55.391658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:55.393132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:55.445062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c4c9-1f6d-452f-a419-b33fe656c72f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:55.445354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c4c9-1f6d-452f-a419-b33fe656c72f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:55.445640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c4c9-1f6d-452f-a419-b33fe656c72f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:55.447032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 162/322] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:15:55.451961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe084c9-ecdd-4e5a-9fea-b09f4dcb4e9a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:55.452251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe084c9-ecdd-4e5a-9fea-b09f4dcb4e9a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:55.452564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe084c9-ecdd-4e5a-9fea-b09f4dcb4e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:55.471233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dbe084c9-ecdd-4e5a-9fea-b09f4dcb4e9a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:55.473195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dbe084c9-ecdd-4e5a-9fea-b09f4dcb4e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:55.497069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 161/323] 10.222.0.128 () {60 vars in 1139 bytes} [Tue Jul 26 15:15:55 2022] POST /identity/v3/groups => generated 227 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:15:55.557685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 163/324] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:55.580081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:55.612670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:55.692405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39436a-80b4-4b60-a61c-85d39cc36cc7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:55.728150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:55.728761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:55.729049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:55.729390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:55.741545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:55.747488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:55.748404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:15:55.749279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 162/325] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:55.755376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe084c9-ecdd-4e5a-9fea-b09f4dcb4e9a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:55.782889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:55.872176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe084c9-ecdd-4e5a-9fea-b09f4dcb4e9a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:55.912893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13410a-7cdb-4e79-9bdd-badafff8c537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:55.913902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13410a-7cdb-4e79-9bdd-badafff8c537 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:55.914389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13410a-7cdb-4e79-9bdd-badafff8c537 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:55.914865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13410a-7cdb-4e79-9bdd-badafff8c537 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:55.915823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13410a-7cdb-4e79-9bdd-badafff8c537 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:55.916353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13410a-7cdb-4e79-9bdd-badafff8c537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:55.917982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13410a-7cdb-4e79-9bdd-badafff8c537 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:55.928718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 164/326] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:55 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:55.933931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:55.995632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57343189-e847-43df-a39e-f25b35315596 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:55.996803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57343189-e847-43df-a39e-f25b35315596 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:55.997300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57343189-e847-43df-a39e-f25b35315596 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:55.999131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 165/327] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:55 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:56.006347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aaa2b0-9c05-4060-bb16-27ee62fc6a4a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:56.006894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aaa2b0-9c05-4060-bb16-27ee62fc6a4a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:56.007419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aaa2b0-9c05-4060-bb16-27ee62fc6a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:56.027878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-41aaa2b0-9c05-4060-bb16-27ee62fc6a4a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:56.029477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-41aaa2b0-9c05-4060-bb16-27ee62fc6a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:56.054966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:56.071035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d473b-dbdf-4d10-bdbc-dd56ed5892c8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:56.119361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:56.120918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:56.121704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:56.122214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] PATH_INFO: `/v3/users/37814491ff224d2e95f79a5147d44c44` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:56.124140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] RBAC: Authorizing `identity:get_user(user_id=37814491ff224d2e95f79a5147d44c44)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:56.126264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:56.129712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 163/328] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:55 2022] GET /identity/v3/users/37814491ff224d2e95f79a5147d44c44 => generated 294 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:56.135330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:56.141302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 166/329] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:56.171600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aaa2b0-9c05-4060-bb16-27ee62fc6a4a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:56.192813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:56.273374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81a95e-7ab0-43ac-8c31-546d50eb6ddd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:56.312110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:56.313221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:56.313703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:56.314251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] PATH_INFO: `/v3/system/users/37814491ff224d2e95f79a5147d44c44/roles/a9b476e637814e0f8b3a8aa8e5ca9578` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:56.321445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=37814491ff224d2e95f79a5147d44c44, role_id=a9b476e637814e0f8b3a8aa8e5ca9578)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:56.323177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:56.345765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aaa2b0-9c05-4060-bb16-27ee62fc6a4a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:56.385874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:56.386932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:56.387408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:56.387873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:56.400458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:56.405990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:56.406845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:56.407628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:56.411159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 164/330] 10.222.0.128 () {58 vars in 1339 bytes} [Tue Jul 26 15:15:56 2022] PUT /identity/v3/system/users/37814491ff224d2e95f79a5147d44c44/roles/a9b476e637814e0f8b3a8aa8e5ca9578 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:15:56.413202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 167/331] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:56.419406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:56.447416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:56.554306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0bc32-65ca-43b1-990f-db64fbfb23f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:56.603430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bb189-7478-4802-9309-b135b138e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:56.604447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bb189-7478-4802-9309-b135b138e16e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:56.604977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bb189-7478-4802-9309-b135b138e16e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:56.605476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bb189-7478-4802-9309-b135b138e16e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:56.606404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bb189-7478-4802-9309-b135b138e16e None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:56.606916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bb189-7478-4802-9309-b135b138e16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:56.608638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bb189-7478-4802-9309-b135b138e16e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:56.619940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 165/332] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:56 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:56.625891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:56.711638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:56.745944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdaefa7-6847-4d10-8a3b-325dd6c67565 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:56.783194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0582218-bd05-45d6-b3ff-2c2c9ce2fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:56.784937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582218-bd05-45d6-b3ff-2c2c9ce2fc3e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:56.785649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582218-bd05-45d6-b3ff-2c2c9ce2fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:56.786145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582218-bd05-45d6-b3ff-2c2c9ce2fc3e None admin] PATH_INFO: `/v3/users/0876821669e74bed8463e9ae4a32ec9f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:56.787587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0582218-bd05-45d6-b3ff-2c2c9ce2fc3e None admin] RBAC: Authorizing `identity:get_user(user_id=0876821669e74bed8463e9ae4a32ec9f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:56.789309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0582218-bd05-45d6-b3ff-2c2c9ce2fc3e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:56.791422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 168/333] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:15:56 2022] GET /identity/v3/users/0876821669e74bed8463e9ae4a32ec9f => generated 294 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:56.797450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bb189-7478-4802-9309-b135b138e16e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:56.894206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:56.914727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bb189-7478-4802-9309-b135b138e16e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:56.953456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451b352-8cc6-4764-bddb-2ed36d22f1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:56.954081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451b352-8cc6-4764-bddb-2ed36d22f1fc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:56.954363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451b352-8cc6-4764-bddb-2ed36d22f1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:56.954643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451b352-8cc6-4764-bddb-2ed36d22f1fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:56.955446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451b352-8cc6-4764-bddb-2ed36d22f1fc None admin] RBAC: Adding query filter params (role.id=dc7c0f1a0476470daff325a7c7adf5f2, user.id=0876821669e74bed8463e9ae4a32ec9f, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:56.955803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451b352-8cc6-4764-bddb-2ed36d22f1fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:56.957376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451b352-8cc6-4764-bddb-2ed36d22f1fc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:56.973749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 166/334] 10.222.0.128 () {56 vars in 1313 bytes} [Tue Jul 26 15:15:56 2022] GET /identity/v3/role_assignments?user.id=0876821669e74bed8463e9ae4a32ec9f&scope.system=all&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 512 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:58.280452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e20479-9c05-4936-b810-bffa6d8a3205 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:58.281540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e20479-9c05-4936-b810-bffa6d8a3205 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:58.282046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e20479-9c05-4936-b810-bffa6d8a3205 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:58.284701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 169/335] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:58 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:58.289945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24a977-a8a6-4c25-8001-dccbba10821e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:58.290968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24a977-a8a6-4c25-8001-dccbba10821e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:58.291455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24a977-a8a6-4c25-8001-dccbba10821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:58.311909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4e24a977-a8a6-4c25-8001-dccbba10821e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:58.313490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4e24a977-a8a6-4c25-8001-dccbba10821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:58.395965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 167/336] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:58.438466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e20479-9c05-4936-b810-bffa6d8a3205 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:58.572681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e20479-9c05-4936-b810-bffa6d8a3205 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:58.608700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61c548-b3d3-4183-9a1e-55519bb261b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:58.609716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c548-b3d3-4183-9a1e-55519bb261b9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:58.610222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c548-b3d3-4183-9a1e-55519bb261b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:58.610726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c548-b3d3-4183-9a1e-55519bb261b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:58.619874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:58.622698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61c548-b3d3-4183-9a1e-55519bb261b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-aa61c548-b3d3-4183-9a1e-55519bb261b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:58.628609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:58.629558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:58.630542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 170/337] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:15:58.635611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24a977-a8a6-4c25-8001-dccbba10821e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:58.749571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:58.779771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f89e-804f-47ea-874a-bda828c58304 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:58.780093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f89e-804f-47ea-874a-bda828c58304 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:58.780307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f89e-804f-47ea-874a-bda828c58304 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:58.781932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 171/338] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:58.782424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24a977-a8a6-4c25-8001-dccbba10821e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:58.799028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cdae3-52b5-41a9-8725-7835a4a18419 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:58.799353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cdae3-52b5-41a9-8725-7835a4a18419 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:58.800144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cdae3-52b5-41a9-8725-7835a4a18419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:58.834575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2c4cdae3-52b5-41a9-8725-7835a4a18419 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:58.834985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:58.835638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:58.835752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2c4cdae3-52b5-41a9-8725-7835a4a18419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:58.835929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:58.836200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:58.836972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:58.837284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:58.838745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:58.855950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 168/339] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:15:58 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:58.866264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:58.927255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 172/340] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:15:58.952692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cdae3-52b5-41a9-8725-7835a4a18419 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:59.063692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cdae3-52b5-41a9-8725-7835a4a18419 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:59.097804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:59.103101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:59.103763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.104041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.104319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.117200 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:15:59.127567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:15:59.128533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:15:59.130102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 173/341] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:15:59.130766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:59.143416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12545a-7d80-4008-a2ab-02ef8c0ff134 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:59.224358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:59.241898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a6a7a-a4fe-4896-867a-8cd4d1f06072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:59.242547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a6a7a-a4fe-4896-867a-8cd4d1f06072 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.242810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50ee69-fe4b-4720-84af-89885d194e04 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:59.242919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a6a7a-a4fe-4896-867a-8cd4d1f06072 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.243100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a6a7a-a4fe-4896-867a-8cd4d1f06072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.390909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb16634-b707-4a8d-9d04-9996a5a0ebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:59.391579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16634-b707-4a8d-9d04-9996a5a0ebc5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.391808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16634-b707-4a8d-9d04-9996a5a0ebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.392021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16634-b707-4a8d-9d04-9996a5a0ebc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.392813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb16634-b707-4a8d-9d04-9996a5a0ebc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:15:59.393658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb16634-b707-4a8d-9d04-9996a5a0ebc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:59.395096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb16634-b707-4a8d-9d04-9996a5a0ebc5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:59.412710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 174/342] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:15:59 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:15:59.414178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6570f-f54d-43ad-8d41-a9f5988e6c99 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.414473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6570f-f54d-43ad-8d41-a9f5988e6c99 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.414749 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6570f-f54d-43ad-8d41-a9f5988e6c99 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.416393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 175/343] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:59.417560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b1bb-2780-47bf-8d75-25f894941070 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.417846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b1bb-2780-47bf-8d75-25f894941070 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.418125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b1bb-2780-47bf-8d75-25f894941070 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.419593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 176/344] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:15:59 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:15:59.426884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aa5fb-97e5-47fd-b037-360a52fa7cef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.427178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aa5fb-97e5-47fd-b037-360a52fa7cef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.427458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aa5fb-97e5-47fd-b037-360a52fa7cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.460427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d36aa5fb-97e5-47fd-b037-360a52fa7cef None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:59.461875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d36aa5fb-97e5-47fd-b037-360a52fa7cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:59.464604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a6a7a-a4fe-4896-867a-8cd4d1f06072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:59.467180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a6a7a-a4fe-4896-867a-8cd4d1f06072 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:59.531169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:59.554927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 177/345] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:59.555800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36aa5fb-97e5-47fd-b037-360a52fa7cef None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:59.595519 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 169/346] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:15:59.601352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af76f6b-7d2e-412f-80e9-b044a7221c4c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.601827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af76f6b-7d2e-412f-80e9-b044a7221c4c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.602276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af76f6b-7d2e-412f-80e9-b044a7221c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.627063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2af76f6b-7d2e-412f-80e9-b044a7221c4c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:59.629852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2af76f6b-7d2e-412f-80e9-b044a7221c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:59.666095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:59.686193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36aa5fb-97e5-47fd-b037-360a52fa7cef None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:59.721846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 170/347] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:15:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:15:59.722621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af76f6b-7d2e-412f-80e9-b044a7221c4c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:59.741597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:59.742294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.742576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.742852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.806772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:59.828097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:59.829877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:59.873471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af76f6b-7d2e-412f-80e9-b044a7221c4c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:59.929352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:59.930754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:59.931053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:59.931336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:59.933359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:59.934824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:59.937436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 171/348] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:15:59 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:15:59.938299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:59.970074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 178/349] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:15:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:15:59.971797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:59.988726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.070401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3fb7b-afc0-4b9d-a6b7-5f6104782bbe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.075375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.100271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa3f68-ecd1-4ea0-bffe-719ac91815b4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.110424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.111265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.112230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.112702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.126856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:00.132369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:00.133381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:00.134187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 172/350] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:15:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:00.134872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.157783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c02997-cd6d-4769-ba91-5deb0144593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.159266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c02997-cd6d-4769-ba91-5deb0144593b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.159553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c02997-cd6d-4769-ba91-5deb0144593b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.159827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c02997-cd6d-4769-ba91-5deb0144593b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.172788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c02997-cd6d-4769-ba91-5deb0144593b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d2c02997-cd6d-4769-ba91-5deb0144593b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:00.178613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:00.179603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:00.180425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:00.181789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 179/351] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:15:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:00.184302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad233-a8ab-43c0-952c-18b4a232a489 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.187532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad233-a8ab-43c0-952c-18b4a232a489 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.187532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad233-a8ab-43c0-952c-18b4a232a489 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.187532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 180/352] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:00.187532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ad233-a8ab-43c0-952c-18b4a232a489 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.268669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.284184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4f94-8d05-4f84-8083-377839504fd5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.333267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.334598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.335113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.335663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.341141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ad233-a8ab-43c0-952c-18b4a232a489 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.346749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:00.348412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:00.350396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 173/353] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:00.350934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.390400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.391422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.392532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.393070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.394833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:00.396313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:00.398693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 181/354] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:00.398921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.412992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.496248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf4e9-8f09-42fb-97bf-a9417f237453 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.516744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55da12-2bfc-4702-9e27-eec4705ca29c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.534753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93ac62-fcc7-4d16-a6e9-7ee4f7b6eedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.535387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ac62-fcc7-4d16-a6e9-7ee4f7b6eedd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.535677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ac62-fcc7-4d16-a6e9-7ee4f7b6eedd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.535962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ac62-fcc7-4d16-a6e9-7ee4f7b6eedd None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.537534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93ac62-fcc7-4d16-a6e9-7ee4f7b6eedd None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:00.537534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93ac62-fcc7-4d16-a6e9-7ee4f7b6eedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:00.538456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93ac62-fcc7-4d16-a6e9-7ee4f7b6eedd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:00.548286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 174/355] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:00.549506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80e33-2f59-4993-bf8c-2455dd86d143 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.549796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80e33-2f59-4993-bf8c-2455dd86d143 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.550170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80e33-2f59-4993-bf8c-2455dd86d143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.556965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.557592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.557874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.558144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.558892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:00.559189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:00.560541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:00.574775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 182/356] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:00.576127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.577448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1bf80e33-2f59-4993-bf8c-2455dd86d143 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:00.578655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1bf80e33-2f59-4993-bf8c-2455dd86d143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:00.593052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.664061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.668746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 175/357] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:00.669446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80e33-2f59-4993-bf8c-2455dd86d143 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.706874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23f490-41ee-4c1f-8bfd-28812f667afa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.749535 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab82b4-fb6c-42ef-b517-f652e6f89a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.750082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab82b4-fb6c-42ef-b517-f652e6f89a2e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.750366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab82b4-fb6c-42ef-b517-f652e6f89a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.750637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab82b4-fb6c-42ef-b517-f652e6f89a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.751809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab82b4-fb6c-42ef-b517-f652e6f89a2e None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=fec0664d0d794aee9dc2952586baef2b) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:00.751809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab82b4-fb6c-42ef-b517-f652e6f89a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:00.753528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab82b4-fb6c-42ef-b517-f652e6f89a2e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:00.759456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 183/358] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 261 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:00.760944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834da624-94d7-43b2-ab54-7f452c317762 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.761237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834da624-94d7-43b2-ab54-7f452c317762 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.761518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834da624-94d7-43b2-ab54-7f452c317762 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.762915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 184/359] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:00.763564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834da624-94d7-43b2-ab54-7f452c317762 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.798380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80e33-2f59-4993-bf8c-2455dd86d143 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.854884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.855578 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.855869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.856152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.857509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:00.859018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:00.860869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 176/360] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:00.862202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:00.883952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.905640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834da624-94d7-43b2-ab54-7f452c317762 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:00.958901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:00.959523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:00.959802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:00.960200 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:00.963945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:00.999913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960e469-b4e3-45ca-92a5-7ebe018bd445 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.044242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f03a1-877e-4e91-b686-13ffd339486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.044749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f03a1-877e-4e91-b686-13ffd339486e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.045035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f03a1-877e-4e91-b686-13ffd339486e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.045314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f03a1-877e-4e91-b686-13ffd339486e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:01.056721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:01.133697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f03a1-877e-4e91-b686-13ffd339486e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:01.135531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f03a1-877e-4e91-b686-13ffd339486e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:01.152310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:01.187947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 185/361] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:01.188983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.241966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:01.268356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 177/362] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:01.268637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91637b4-3b11-44db-9526-b5db529ed719 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.269002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91637b4-3b11-44db-9526-b5db529ed719 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.269287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91637b4-3b11-44db-9526-b5db529ed719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.288241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e91637b4-3b11-44db-9526-b5db529ed719 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:01.289738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e91637b4-3b11-44db-9526-b5db529ed719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:01.345191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403cb5-0910-455d-bc35-43989f948117 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.383407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 178/363] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:01.384645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91637b4-3b11-44db-9526-b5db529ed719 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.396538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.397191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.397549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.397895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.413197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:01.418677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:01.419566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:01.420362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 186/364] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:01.426433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.442369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:01.527215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91637b4-3b11-44db-9526-b5db529ed719 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.540842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:01.548226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9248d-d8e2-4461-8830-2d1afc7d5d07 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.579033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.579686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.580108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.580958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.583998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=fec0664d0d794aee9dc2952586baef2b) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:01.583998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:01.584193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:01.594929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.595118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 179/365] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 261 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:01.595558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.595782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.595880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.596781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.598121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:01.606340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:01.608341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 187/366] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:01.608940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.737762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01f517-9ec5-465f-879d-65f31f24686c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.750511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:01.755545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56e6c-1ee2-4a5b-88e8-868789d317df None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.781185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.782157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.782655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.783124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.786015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:01.787848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:01.790213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 180/367] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:01.791215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.803685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.803685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.803964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.804219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.817577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:01.822580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:01.823490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:01.824727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 188/368] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:01.829069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.857839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:01.917354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90e681-45dd-4aef-a9ea-d9d360a502ef None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.944028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa5ca3-6fc6-4f5f-82fb-b5a6e64f054b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:01.962095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.963067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.963570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.964062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.965020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:01.965522 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:01.967134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:01.978052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 181/369] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:01.979261 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:01.984896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:01.985555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:01.985845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:01.986129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:01.987906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:01.989391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:01.994312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 189/370] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:01.994312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:02.034351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:02.097312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:02.106659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6ab2-9746-4ab7-8256-033e6408f39c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:02.107333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc914b-0999-44b0-8158-a338b06fad1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:02.145331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:02.146507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.147041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.147352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:02.147782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.147984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.148273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.148596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.149337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:02.149650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:02.150172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:02.151030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:02.152123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:02.154300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 182/371] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:02.155259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:02.164567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 190/372] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:01 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:02.170905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:02.263677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928346-dd20-4947-9d0a-9c380d7967da None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:02.271726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:02.287982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eefec-27e7-4acf-a7b1-993d03f07920 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:02.302702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:02.304041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.304692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.305462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] PATH_INFO: `/v3/users/37814491ff224d2e95f79a5147d44c44` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.307118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] RBAC: Authorizing `identity:get_user(user_id=37814491ff224d2e95f79a5147d44c44)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:02.308689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:02.311033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 183/373] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:16:02 2022] GET /identity/v3/users/37814491ff224d2e95f79a5147d44c44 => generated 294 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:02.312040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:02.324053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:02.325134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.325625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.326106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.327234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:02.327757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:02.329600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:02.336930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 191/374] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:02 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 261 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:02.338066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:02.381075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:02.506240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcff353-9542-4252-949a-dfac9adf3f66 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:02.567048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea8a97-6ad0-485b-b654-a0d80078e0ec None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:02.571642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32da03-a71c-4b12-b629-a16a04c61e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:02.572282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32da03-a71c-4b12-b629-a16a04c61e98 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.572605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32da03-a71c-4b12-b629-a16a04c61e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.572885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32da03-a71c-4b12-b629-a16a04c61e98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.633172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:02.633805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.634086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.634366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.635162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:02.635477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:02.636942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:02.650032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 192/375] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:02 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 261 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:02.652910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:02.654419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:02.717687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32da03-a71c-4b12-b629-a16a04c61e98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:02.719213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32da03-a71c-4b12-b629-a16a04c61e98 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:02.825686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:02.866028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c718929-cb1a-470a-acd5-abc87981a9bd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:02.899795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 184/376] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:02.901139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bdea3-1cc5-4d8c-a70f-0caec5ccbc64 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.901427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bdea3-1cc5-4d8c-a70f-0caec5ccbc64 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.901706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bdea3-1cc5-4d8c-a70f-0caec5ccbc64 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.903219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 185/377] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:02.906348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bdea3-1cc5-4d8c-a70f-0caec5ccbc64 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:02.925141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af1276-7144-48b7-b503-dd9fecc356e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:02.925791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af1276-7144-48b7-b503-dd9fecc356e7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.926074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af1276-7144-48b7-b503-dd9fecc356e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.926353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af1276-7144-48b7-b503-dd9fecc356e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.927698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af1276-7144-48b7-b503-dd9fecc356e7 None admin] RBAC: Adding query filter params (role.id=a9b476e637814e0f8b3a8aa8e5ca9578, user.id=37814491ff224d2e95f79a5147d44c44, scope.system=all) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:02.928018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af1276-7144-48b7-b503-dd9fecc356e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:02.929532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af1276-7144-48b7-b503-dd9fecc356e7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:02.943207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 193/378] 10.222.0.128 () {56 vars in 1313 bytes} [Tue Jul 26 15:16:02 2022] GET /identity/v3/role_assignments?user.id=37814491ff224d2e95f79a5147d44c44&scope.system=all&role.id=a9b476e637814e0f8b3a8aa8e5ca9578 => generated 512 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:02.944734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d708b-3de9-42e7-981a-22573fbddef7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:02.945029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d708b-3de9-42e7-981a-22573fbddef7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:02.945311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d708b-3de9-42e7-981a-22573fbddef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:02.965357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-606d708b-3de9-42e7-981a-22573fbddef7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:02.966744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-606d708b-3de9-42e7-981a-22573fbddef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:03.048079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 194/379] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:03.071368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d708b-3de9-42e7-981a-22573fbddef7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:03.094250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:03.147992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:03.153251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bdea3-1cc5-4d8c-a70f-0caec5ccbc64 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:03.179143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d708b-3de9-42e7-981a-22573fbddef7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:03.217039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:03.217755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:03.218042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:03.218329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:03.221411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:03.222104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:03.222392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:03.222672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] PATH_INFO: `/v3/projects/b15b339f34b0459888ba2e587e895d4f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:03.223984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] RBAC: Authorizing `identity:get_project(project_id=b15b339f34b0459888ba2e587e895d4f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:03.225583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:03.227607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 186/380] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:02 2022] GET /identity/v3/projects/b15b339f34b0459888ba2e587e895d4f => generated 345 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:03.230566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:03.233190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:03.236287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:03.237926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:03.238717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:03.239939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 195/381] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:03.250874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:03.358704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff6c5c-a376-45e0-8294-67b8482d4964 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:03.409011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:03.409986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:03.410374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:03.410762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] PATH_INFO: `/v3/groups/209e100236494511ab679c82577fc29e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:03.416281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] RBAC: Authorizing `identity:get_group(group_id=209e100236494511ab679c82577fc29e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:03.418031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:03.420152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 187/382] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:03 2022] GET /identity/v3/groups/209e100236494511ab679c82577fc29e => generated 230 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:03.433450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:03.445856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:03.500360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b5ea0-d130-47ef-946e-2713fd193aa4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:03.551886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7bdfd-11c5-4e78-a3db-f29a34172040 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:03.554794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:03.570119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:03.576746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:03.576746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:03.576746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:03.576746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:03.576746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:03.576746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:03.588668 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:03.589307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:03.589611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:03.589915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:03.590729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] RBAC: Adding query filter params (group.id=209e100236494511ab679c82577fc29e, role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=b15b339f34b0459888ba2e587e895d4f) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:03.591046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:03.592079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 196/383] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:03 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:03.592514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:03.598811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:03.600145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 188/384] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:03 2022] GET /identity/v3/role_assignments?group.id=209e100236494511ab679c82577fc29e&scope.project.id=b15b339f34b0459888ba2e587e895d4f&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 261 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:03.721936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c07a6-c72c-456e-9e88-d55ec4f68860 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:03.762452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:03.763106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:03.763389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:03.763665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:03.764907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:03.766353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:03.772617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 197/385] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:03 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:03.774205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:03.864726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:03.909561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d867977-3be3-4a80-ba36-4c46b824b3e0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:03.953094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321be41-e222-4767-97bc-acd848075626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:03.954153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321be41-e222-4767-97bc-acd848075626 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:03.954656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321be41-e222-4767-97bc-acd848075626 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:03.955138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321be41-e222-4767-97bc-acd848075626 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:03.957207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321be41-e222-4767-97bc-acd848075626 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:03.958878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321be41-e222-4767-97bc-acd848075626 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:03.961253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 189/386] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:03 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:03.966832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:04.042628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:04.093135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e5018-78a7-4c24-b86c-8f5e27f6ce44 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:04.134238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:04.135304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:04.135890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:04.136373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b/groups/169635fef0d94cacbbc28706b2edd09f/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:04.138821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] RBAC: Authorizing `identity:create_grant(project_id=fec0664d0d794aee9dc2952586baef2b, group_id=169635fef0d94cacbbc28706b2edd09f, role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:04.140529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:04.145322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ea02-1312-4472-9564-e0e50be1eaef None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:04.145909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ea02-1312-4472-9564-e0e50be1eaef None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:04.146574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ea02-1312-4472-9564-e0e50be1eaef None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:04.148249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 190/387] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:04 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:04.153303 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ed3f-908e-45bd-8a56-4b7ee5b9f8ef None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:04.153883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ed3f-908e-45bd-8a56-4b7ee5b9f8ef None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:04.154387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ed3f-908e-45bd-8a56-4b7ee5b9f8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:04.177796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-44b6ed3f-908e-45bd-8a56-4b7ee5b9f8ef None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:04.179412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-44b6ed3f-908e-45bd-8a56-4b7ee5b9f8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:04.256972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 191/388] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:04.261668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 198/389] 10.222.0.128 () {58 vars in 1447 bytes} [Tue Jul 26 15:16:03 2022] PUT /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b/groups/169635fef0d94cacbbc28706b2edd09f/roles/364a95d0748e48e9a538b4e6be293f91 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:04.288214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ed3f-908e-45bd-8a56-4b7ee5b9f8ef None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:04.378390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:04.428390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ed3f-908e-45bd-8a56-4b7ee5b9f8ef None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:04.489151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:04.489973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:04.490364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:04.490746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:04.507119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:04.512497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:04.513440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:04.514865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 192/390] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:04.520134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:04.638247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd75eba-18a9-42bf-87e7-5429532da143 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:04.661496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:04.677003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:04.680701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:04.680701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:04.680701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:04.680701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:04.680701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:04.681039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:04.690238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 199/391] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:16:04 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:04.701790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:04.853666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ce79a-e9a3-414c-9dc7-5a304184716c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:04.891576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:04.900412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:04.901099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:04.901387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:04.901670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:04.902905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:04.904488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:04.906413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 193/392] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:04 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:04.913590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:05.061354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d0bd-9a01-4f20-b58e-bbc6ad9dfcae None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:05.114812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043948-9da5-4db5-90bc-9eee7b864c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:05.115475 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043948-9da5-4db5-90bc-9eee7b864c79 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.115756 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043948-9da5-4db5-90bc-9eee7b864c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.116032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043948-9da5-4db5-90bc-9eee7b864c79 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.117443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043948-9da5-4db5-90bc-9eee7b864c79 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:05.119168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043948-9da5-4db5-90bc-9eee7b864c79 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:05.121171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 200/393] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:04 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:05.128195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:05.178073 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77fb44-ed67-4f08-b6ee-a89570576678 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.178316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77fb44-ed67-4f08-b6ee-a89570576678 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.178562 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77fb44-ed67-4f08-b6ee-a89570576678 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.180496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 201/394] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:05.187070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06178ca1-d5c8-4fec-99e2-d62a3dd3df04 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.187531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06178ca1-d5c8-4fec-99e2-d62a3dd3df04 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.187815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06178ca1-d5c8-4fec-99e2-d62a3dd3df04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.223429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-06178ca1-d5c8-4fec-99e2-d62a3dd3df04 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:05.224785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-06178ca1-d5c8-4fec-99e2-d62a3dd3df04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:05.290216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3a759-ec65-47e5-8d4b-bf43ddd0dc42 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:05.326617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:05.359101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:05.359800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.360186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.360486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b/groups/169635fef0d94cacbbc28706b2edd09f/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.363995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] RBAC: Authorizing `identity:create_grant(project_id=fec0664d0d794aee9dc2952586baef2b, group_id=169635fef0d94cacbbc28706b2edd09f, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:05.365478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:05.369321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 202/395] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:16:05.394201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06178ca1-d5c8-4fec-99e2-d62a3dd3df04 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:05.442797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 194/396] 10.222.0.128 () {58 vars in 1447 bytes} [Tue Jul 26 15:16:05 2022] PUT /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b/groups/169635fef0d94cacbbc28706b2edd09f/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:16:05.558303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06178ca1-d5c8-4fec-99e2-d62a3dd3df04 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:05.611170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ca0a1-9148-452f-a02a-d41825295eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:05.612672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ca0a1-9148-452f-a02a-d41825295eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.613054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ca0a1-9148-452f-a02a-d41825295eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.613338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ca0a1-9148-452f-a02a-d41825295eb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.627987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ca0a1-9148-452f-a02a-d41825295eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f42ca0a1-9148-452f-a02a-d41825295eb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:05.633186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:05.634114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:05.635384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 203/397] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:05.640568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:05.686516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b68667-5ae5-4a6e-bab6-a614a4b7a7c4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.687065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b68667-5ae5-4a6e-bab6-a614a4b7a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.687065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b68667-5ae5-4a6e-bab6-a614a4b7a7c4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.689134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 204/398] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:05.695787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8711143-a0e9-4b50-9139-8afaf02b9f4b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.696061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8711143-a0e9-4b50-9139-8afaf02b9f4b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.696293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8711143-a0e9-4b50-9139-8afaf02b9f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.717136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:05.719332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b8711143-a0e9-4b50-9139-8afaf02b9f4b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:05.720776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b8711143-a0e9-4b50-9139-8afaf02b9f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:05.763352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:05.792750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd9496-53b8-4aa6-aac3-c1e700d62727 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:05.830837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 205/399] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:05.833546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:05.834433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:05.834860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:05.835277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:05.836164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:05.836642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:05.838135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:05.848164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 195/400] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:16:05 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:05.853749 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8711143-a0e9-4b50-9139-8afaf02b9f4b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:05.868272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:05.987190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942fe4f-2781-4784-89ba-3de1e92097b1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:06.008132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:06.029354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:06.030357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.030867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.031341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.038724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8711143-a0e9-4b50-9139-8afaf02b9f4b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:06.047524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:06.053864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:06.054837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:06.055637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:06.058275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 196/401] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:06.064580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:06.082281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:06.082944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.083347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.083632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.086637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:06.088088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:06.090366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 206/402] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:05 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:06.095041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:06.170397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:06.271895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095ea7c-210a-425c-9664-c03df603d7f5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:06.296648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00516e94-9347-4a60-863c-a57463f5cd89 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:06.340775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:06.341415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.341703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.341983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.342714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:06.343027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:06.344409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:06.351213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:06.351867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.352148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.352421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.353967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:06.358625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:06.360329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 207/403] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:06 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:06.364563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 197/404] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:06 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:06.369917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:06.370125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:06.436384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:06.506282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566af82-21fc-4b94-9763-bd1e10f241fe None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:06.512528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572c1d-03e5-4f75-ae23-41e6e830884c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:06.514613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:06.552700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c63985-4f66-4c7e-a1fd-c8797a2face7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:06.553268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c63985-4f66-4c7e-a1fd-c8797a2face7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.553549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c63985-4f66-4c7e-a1fd-c8797a2face7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.553825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c63985-4f66-4c7e-a1fd-c8797a2face7 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.555353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c63985-4f66-4c7e-a1fd-c8797a2face7 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:06.556975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c63985-4f66-4c7e-a1fd-c8797a2face7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:06.558353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429cf18-ba60-4899-aa92-11a98d73cb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:06.559254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429cf18-ba60-4899-aa92-11a98d73cb94 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.559808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429cf18-ba60-4899-aa92-11a98d73cb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.560288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429cf18-ba60-4899-aa92-11a98d73cb94 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/groups/169635fef0d94cacbbc28706b2edd09f/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.562512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429cf18-ba60-4899-aa92-11a98d73cb94 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353, group_id=169635fef0d94cacbbc28706b2edd09f, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:06.563066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 198/405] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:06 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:06.564272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115dae2-2781-49d5-95d1-f5db096f96c2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.565183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115dae2-2781-49d5-95d1-f5db096f96c2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.565470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115dae2-2781-49d5-95d1-f5db096f96c2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.565814 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429cf18-ba60-4899-aa92-11a98d73cb94 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:06.566724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 199/406] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:06.567414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115dae2-2781-49d5-95d1-f5db096f96c2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:06.667977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 208/407] 10.222.0.128 () {58 vars in 1447 bytes} [Tue Jul 26 15:16:06 2022] PUT /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/groups/169635fef0d94cacbbc28706b2edd09f/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:06.669781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05874849-38df-463e-b646-4254838b059a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.670325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05874849-38df-463e-b646-4254838b059a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.671502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05874849-38df-463e-b646-4254838b059a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.694563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115dae2-2781-49d5-95d1-f5db096f96c2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:06.697297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-05874849-38df-463e-b646-4254838b059a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:06.699175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-05874849-38df-463e-b646-4254838b059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:06.713453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:06.763678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:06.764570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:06.764763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:06.765014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:06.766287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:06.768008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:06.777204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 200/408] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:06 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:06.778154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:06.795463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 209/409] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:06.847052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05874849-38df-463e-b646-4254838b059a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:06.967760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23998852-a3f5-4167-8770-8d7836f38f7a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:07.016194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:07.020740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900a87d-5b63-49f2-8bdc-c28f154b550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:07.021640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900a87d-5b63-49f2-8bdc-c28f154b550d None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.021936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900a87d-5b63-49f2-8bdc-c28f154b550d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.022334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900a87d-5b63-49f2-8bdc-c28f154b550d None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/groups/169635fef0d94cacbbc28706b2edd09f/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.023921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05874849-38df-463e-b646-4254838b059a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:07.026904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900a87d-5b63-49f2-8bdc-c28f154b550d None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353, group_id=169635fef0d94cacbbc28706b2edd09f, role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:07.028384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900a87d-5b63-49f2-8bdc-c28f154b550d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:07.071532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ba6b8-4da1-4a4b-880d-b103c0e7cafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:07.072193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba6b8-4da1-4a4b-880d-b103c0e7cafa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.072500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba6b8-4da1-4a4b-880d-b103c0e7cafa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.072778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba6b8-4da1-4a4b-880d-b103c0e7cafa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.085649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ba6b8-4da1-4a4b-880d-b103c0e7cafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-dd6ba6b8-4da1-4a4b-880d-b103c0e7cafa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:07.091119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:07.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:07.093616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 210/410] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:07.094282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c3745-166f-493f-afaa-7337690985a3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.094282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c3745-166f-493f-afaa-7337690985a3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.101585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c3745-166f-493f-afaa-7337690985a3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.101585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 211/411] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:07.101585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c3745-166f-493f-afaa-7337690985a3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:07.135657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 201/412] 10.222.0.128 () {58 vars in 1447 bytes} [Tue Jul 26 15:16:06 2022] PUT /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353/groups/169635fef0d94cacbbc28706b2edd09f/roles/364a95d0748e48e9a538b4e6be293f91 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:07.142015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9b53c-7dfa-4383-b73d-95974b6cc292 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.142015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9b53c-7dfa-4383-b73d-95974b6cc292 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.142015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9b53c-7dfa-4383-b73d-95974b6cc292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.178621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9bb9b53c-7dfa-4383-b73d-95974b6cc292 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:07.179871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9bb9b53c-7dfa-4383-b73d-95974b6cc292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:07.261564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:07.308015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c3745-166f-493f-afaa-7337690985a3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:07.359552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 202/413] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:07.364264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:07.364938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.365224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.365501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.366289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:07.366601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:07.370009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:07.379792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 212/414] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:07.385454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9b53c-7dfa-4383-b73d-95974b6cc292 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:07.390318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:07.419061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:07.516513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66371008-6337-4604-8cfa-b91fd675ab69 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:07.550752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9b53c-7dfa-4383-b73d-95974b6cc292 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:07.566387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:07.567112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.567396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.567713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.589984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:07.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:07.601799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:07.602641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:07.616752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 213/415] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:07.617573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:07.637984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:07.638651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.638927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.639201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] PATH_INFO: `/v3/groups/209e100236494511ab679c82577fc29e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.640545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] RBAC: Authorizing `identity:get_group(group_id=209e100236494511ab679c82577fc29e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:07.642023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:07.644371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 203/416] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:07 2022] GET /identity/v3/groups/209e100236494511ab679c82577fc29e => generated 230 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:07.650119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:07.737072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:07.738770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:07.762361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19565-c8cf-4770-b60c-58ad170e27f3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:07.813247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:07.813886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.814168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.814498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.815376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:07.815690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:07.818233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:07.835867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 214/417] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:07 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:07.841360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:07.847935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3525d8-7b9e-4116-99a8-95c2b7f136ab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:07.898597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:07.899171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:07.899452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:07.899732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] PATH_INFO: `/v3/projects/b15b339f34b0459888ba2e587e895d4f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:07.901246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] RBAC: Authorizing `identity:get_project(project_id=b15b339f34b0459888ba2e587e895d4f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:07.902899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:07.905183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 204/418] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:07 2022] GET /identity/v3/projects/b15b339f34b0459888ba2e587e895d4f => generated 345 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:07.911846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:08.000086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4784e5f-6013-4ec8-a9b1-15835b39f8d5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:08.041868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db41ee-559e-4319-a20f-c8e60233fe87 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:08.044294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:08.047238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:08.047876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.048163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.048439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.063575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:08.086309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c500f-19d8-4995-9397-2dc297bfb712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:08.087872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c500f-19d8-4995-9397-2dc297bfb712 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.088312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c500f-19d8-4995-9397-2dc297bfb712 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.088757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c500f-19d8-4995-9397-2dc297bfb712 None admin] PATH_INFO: `/v3/projects/b15b339f34b0459888ba2e587e895d4f/groups/209e100236494511ab679c82577fc29e/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.091933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c500f-19d8-4995-9397-2dc297bfb712 None admin] RBAC: Authorizing `identity:create_grant(project_id=b15b339f34b0459888ba2e587e895d4f, group_id=209e100236494511ab679c82577fc29e, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:08.093663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c500f-19d8-4995-9397-2dc297bfb712 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:08.131124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:08.132932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:08.176373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 205/419] 10.222.0.128 () {58 vars in 1447 bytes} [Tue Jul 26 15:16:07 2022] PUT /identity/v3/projects/b15b339f34b0459888ba2e587e895d4f/groups/209e100236494511ab679c82577fc29e/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:08.178746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52308cf3-0be7-4acc-ad9f-efb50e6f6586 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.179716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52308cf3-0be7-4acc-ad9f-efb50e6f6586 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.180553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52308cf3-0be7-4acc-ad9f-efb50e6f6586 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.182565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 206/420] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:08.198115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742d205-761b-43e2-812a-2d95c92ad853 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.198413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742d205-761b-43e2-812a-2d95c92ad853 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.198687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742d205-761b-43e2-812a-2d95c92ad853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.241487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9742d205-761b-43e2-812a-2d95c92ad853 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:08.242705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9742d205-761b-43e2-812a-2d95c92ad853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:08.251053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 215/421] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:07 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:16:08.257155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:08.329373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:08.401185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 207/422] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:08.402484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca94b38-742c-4066-82fe-d8ed4dbc4785 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:08.428720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742d205-761b-43e2-812a-2d95c92ad853 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:08.449690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:08.450679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.451158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.451646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.453084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:08.454785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:08.462890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 216/423] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:08 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:08.466971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:08.589176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:08.596825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742d205-761b-43e2-812a-2d95c92ad853 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:08.634748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6b51-ddc1-4d20-b992-820d6400fd8d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:08.635632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:08.636485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.636778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.637056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.648867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:08.650978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:08.654871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:08.655783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:08.656617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:08.662023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 208/424] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:08.670078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:08.716129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:08.716818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.717158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.717437 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.718790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:08.720232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:08.727896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 217/425] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:08 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:08.750513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:08.869958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bba4ff-0207-4a60-abab-409af157bd3e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:08.908907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45a3ea-47b6-45f3-ac28-609d9e7b3828 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:08.947286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:08.948180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.948552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.948832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.949685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=fec0664d0d794aee9dc2952586baef2b) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:08.950008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:08.951451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:08.959195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 218/426] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:08 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 613 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:08.981379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbd6e5-aebd-479c-b377-0d34656c0108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:08.981963 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbd6e5-aebd-479c-b377-0d34656c0108 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:08.982255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbd6e5-aebd-479c-b377-0d34656c0108 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:08.982542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbd6e5-aebd-479c-b377-0d34656c0108 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:08.983305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbd6e5-aebd-479c-b377-0d34656c0108 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:08.983627 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbd6e5-aebd-479c-b377-0d34656c0108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:08.985102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbd6e5-aebd-479c-b377-0d34656c0108 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:09.004338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 209/427] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:16:08 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:09.009779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:09.067152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:09.150624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8e8bb-5139-4ad3-b6c1-97689d79543f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:09.186833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:09.187800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.188320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.188817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.244079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42564d6-ceae-4564-9d1f-ced8b7b4230d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.244371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42564d6-ceae-4564-9d1f-ced8b7b4230d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.244587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42564d6-ceae-4564-9d1f-ced8b7b4230d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.246170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 210/428] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:09.251652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1da378-39be-465a-9391-0628e4c927cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.252496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1da378-39be-465a-9391-0628e4c927cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.252790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1da378-39be-465a-9391-0628e4c927cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.263124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:09.264912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:09.274716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9a1da378-39be-465a-9391-0628e4c927cc None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:09.275930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9a1da378-39be-465a-9391-0628e4c927cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:09.286378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:09.316886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:09.355976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 211/429] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:16:09.381180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 219/430] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:09.382358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1da378-39be-465a-9391-0628e4c927cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:09.387839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:09.504480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1da378-39be-465a-9391-0628e4c927cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:09.520757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba064e7-7e28-4ff4-abd1-5b8665ea9d95 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:09.541103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:09.541841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.542034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.542315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.553825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:09.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:09.559757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 15:16:09.560566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:09.564846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 212/431] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:09.566394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:09.568882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:09.570089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.570571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.571834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:09.572706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] PATH_INFO: `/v3/projects/fec0664d0d794aee9dc2952586baef2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.574200 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] RBAC: Authorizing `identity:get_project(project_id=fec0664d0d794aee9dc2952586baef2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:09.575872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:09.578511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 220/432] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:09 2022] GET /identity/v3/projects/fec0664d0d794aee9dc2952586baef2b => generated 299 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:09.584244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:09.602008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:09.689626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ce8cf-9e61-4b85-92b9-717dee904b45 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:09.711721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4b5ca-5ce3-40e8-a9ec-21348c6acf91 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:09.732369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754931d7-41ca-45c5-b3a4-5f19816870c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:09.733838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754931d7-41ca-45c5-b3a4-5f19816870c4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.734403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754931d7-41ca-45c5-b3a4-5f19816870c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.734951 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754931d7-41ca-45c5-b3a4-5f19816870c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.736655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754931d7-41ca-45c5-b3a4-5f19816870c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:09.737193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754931d7-41ca-45c5-b3a4-5f19816870c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:09.738585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754931d7-41ca-45c5-b3a4-5f19816870c4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:09.752437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 213/433] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:16:09 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:09.752654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39278a18-c484-4930-a670-d32190ad1467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:09.752654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39278a18-c484-4930-a670-d32190ad1467 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.752654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39278a18-c484-4930-a670-d32190ad1467 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.753202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c58e2-e427-42e6-a6e3-2b12dbbbee2f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.753272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39278a18-c484-4930-a670-d32190ad1467 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.753338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c58e2-e427-42e6-a6e3-2b12dbbbee2f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.753338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c58e2-e427-42e6-a6e3-2b12dbbbee2f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.753338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 214/434] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:09.754600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39278a18-c484-4930-a670-d32190ad1467 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:09.755845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c58e2-e427-42e6-a6e3-2b12dbbbee2f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:09.756066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39278a18-c484-4930-a670-d32190ad1467 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:09.758037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 221/435] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:09 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:09.759076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4acf5-8965-49d2-882c-2609ac049d16 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.759366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4acf5-8965-49d2-882c-2609ac049d16 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.759643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4acf5-8965-49d2-882c-2609ac049d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.778078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4df4acf5-8965-49d2-882c-2609ac049d16 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:09.779278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4df4acf5-8965-49d2-882c-2609ac049d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:09.831958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:09.852161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 222/436] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:09.853032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4acf5-8965-49d2-882c-2609ac049d16 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:09.872897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:09.881344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c58e2-e427-42e6-a6e3-2b12dbbbee2f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:09.922751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:09.924252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:09.924752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:09.925218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:09.971272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4acf5-8965-49d2-882c-2609ac049d16 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:10.000295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:10.002728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:10.020515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:10.021457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.021457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.021763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.022585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=fec0664d0d794aee9dc2952586baef2b) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:10.022957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:10.024419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:10.031677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 223/437] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:09 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=fec0664d0d794aee9dc2952586baef2b&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 613 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:10.033797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.084690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:10.138274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 215/438] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:10.141136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.151955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:10.187975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04b31c-e6bd-4caf-ab14-1bb0be038b08 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:10.235297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:10.236841 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.237269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.237744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.257398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:10.265782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:10.266787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:10.268330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:10.275296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 224/439] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:10.277674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.289632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a0fcf-78bf-40c0-895d-c58114ededc6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:10.336196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:10.336924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.337206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.337484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.339868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:10.340921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:10.343331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 216/440] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:10.349430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.397087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:10.426521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f67fe2-2cf4-4934-89ff-eb7dc37fb8e8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:10.467819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:10.477725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:10.478367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.478651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.483120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.483120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:10.483120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:10.483120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:10.487391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285c91-41c5-44b5-902b-5ce7f8484fe3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:10.490921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 225/441] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:10.498089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.535274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:10.535461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.535847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.536132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.537461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:10.538924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:10.545853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 217/442] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:10.548285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.627786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7d250-75e4-40b3-b1a4-7a41a225c898 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:10.672030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:10.673387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.673387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.673387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.675176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10f96f-c41f-4d8e-b462-45e6a8e1bc5e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:10.677131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:10.728181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b47a3d-de61-45aa-9a41-1abb4ab11f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:10.728828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47a3d-de61-45aa-9a41-1abb4ab11f7e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.729106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47a3d-de61-45aa-9a41-1abb4ab11f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.729380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47a3d-de61-45aa-9a41-1abb4ab11f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.730170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b47a3d-de61-45aa-9a41-1abb4ab11f7e None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=dc7c0f1a0476470daff325a7c7adf5f2, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:10.730479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b47a3d-de61-45aa-9a41-1abb4ab11f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:10.731903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b47a3d-de61-45aa-9a41-1abb4ab11f7e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:10.738092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 218/443] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=dc7c0f1a0476470daff325a7c7adf5f2 => generated 613 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:10.739256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da697c0-5f9a-4e58-8395-c5c3125cfbb0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.739539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da697c0-5f9a-4e58-8395-c5c3125cfbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.739812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da697c0-5f9a-4e58-8395-c5c3125cfbb0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.741309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 219/444] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:10 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:10.747235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef40d1-5202-489c-94a5-6ac8344dbf50 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:10.747525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef40d1-5202-489c-94a5-6ac8344dbf50 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:10.747810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef40d1-5202-489c-94a5-6ac8344dbf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:10.748038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:10.760917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:10.762538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:10.768575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-93ef40d1-5202-489c-94a5-6ac8344dbf50 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:10.769715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-93ef40d1-5202-489c-94a5-6ac8344dbf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:10.854592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 220/445] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:10.879369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef40d1-5202-489c-94a5-6ac8344dbf50 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.896421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 226/446] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:16:10.902093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:10.998816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:11.007800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef40d1-5202-489c-94a5-6ac8344dbf50 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.020192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:11.026785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db06a9-3102-464a-b1fb-5e348c63a5c7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.051440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.052178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.052484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.052761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.067813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.068321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:11.068746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.069031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.069307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] PATH_INFO: `/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.070496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] RBAC: Authorizing `identity:get_project(project_id=bc7b0b28aeed4c6dae297ea8bfd3b353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:11.071936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:11.073357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:11.074234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:11.075024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:11.075694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 227/447] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/projects/bc7b0b28aeed4c6dae297ea8bfd3b353 => generated 303 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:11.075762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 221/448] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:11.082266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:11.083106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:11.189741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b62cf3-e851-4d27-88c2-473c6acc382e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.190981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8149e-2a65-43a9-91af-fc5692641e8b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.225048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.225255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.225697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.225815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.225981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.226076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.226258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.226358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] PATH_INFO: `/v3/groups/169635fef0d94cacbbc28706b2edd09f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.226981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:11.227374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:11.227575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] RBAC: Authorizing `identity:get_group(group_id=169635fef0d94cacbbc28706b2edd09f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:11.228831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:11.228986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:11.230578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 228/449] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:11 2022] GET /identity/v3/groups/169635fef0d94cacbbc28706b2edd09f => generated 227 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:11.236970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:11.237311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 222/450] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:11 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:11.242734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:11.263155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:11.266623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:11.342469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44a2ea-dff4-4423-9c8c-b8af62a039c8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.344593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0787a-dbbe-4356-aad9-e823f30c5f9e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.378331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.378962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.379257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.379526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.380059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.380320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] RBAC: Adding query filter params (group.id=169635fef0d94cacbbc28706b2edd09f, role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:11.380692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:11.380781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.380985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.381267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.382204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:11.388589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 229/451] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:11 2022] GET /identity/v3/role_assignments?group.id=169635fef0d94cacbbc28706b2edd09f&scope.project.id=bc7b0b28aeed4c6dae297ea8bfd3b353&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 613 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:11.463977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:11.465524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:11.489506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:11.589902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 223/452] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 15:16:11.595671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:11.699562 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:11.709465 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bb838-1543-4851-aff6-6e91f102f406 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.746135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.747019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.747305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.747581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] PATH_INFO: `/v3/projects/b15b339f34b0459888ba2e587e895d4f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.748783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] RBAC: Authorizing `identity:get_project(project_id=b15b339f34b0459888ba2e587e895d4f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:11.750232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:11.752433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 230/453] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:16:11 2022] GET /identity/v3/projects/b15b339f34b0459888ba2e587e895d4f => generated 345 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:11.758089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:11.853934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ec55b-ca8b-4d71-b9b3-ebacec3277f9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:11.874398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:11.890814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a9872-124c-4129-b4eb-b1bed1bb1734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:11.891452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a9872-124c-4129-b4eb-b1bed1bb1734 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:11.891806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a9872-124c-4129-b4eb-b1bed1bb1734 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:11.892106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a9872-124c-4129-b4eb-b1bed1bb1734 None admin] PATH_INFO: `/v3/groups/209e100236494511ab679c82577fc29e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:11.893288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a9872-124c-4129-b4eb-b1bed1bb1734 None admin] RBAC: Authorizing `identity:get_group(group_id=209e100236494511ab679c82577fc29e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:11.894715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a9872-124c-4129-b4eb-b1bed1bb1734 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:11.896305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 224/454] 10.222.0.128 () {56 vars in 1185 bytes} [Tue Jul 26 15:16:11 2022] GET /identity/v3/groups/209e100236494511ab679c82577fc29e => generated 230 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:11.902651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:12.005900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5890aa-e3e7-4153-bd72-6013532f557d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:12.041812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de224a-d3f8-4b9c-9ac5-1ee70e98799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:12.042436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de224a-d3f8-4b9c-9ac5-1ee70e98799e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:12.042721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de224a-d3f8-4b9c-9ac5-1ee70e98799e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:12.043001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de224a-d3f8-4b9c-9ac5-1ee70e98799e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:12.043800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de224a-d3f8-4b9c-9ac5-1ee70e98799e None admin] RBAC: Adding query filter params (group.id=209e100236494511ab679c82577fc29e, role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=b15b339f34b0459888ba2e587e895d4f) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:12.044133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de224a-d3f8-4b9c-9ac5-1ee70e98799e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:12.045601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de224a-d3f8-4b9c-9ac5-1ee70e98799e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:12.051828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 231/455] 10.222.0.128 () {56 vars in 1381 bytes} [Tue Jul 26 15:16:11 2022] GET /identity/v3/role_assignments?group.id=209e100236494511ab679c82577fc29e&scope.project.id=b15b339f34b0459888ba2e587e895d4f&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 613 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:14.597230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67954-3659-49ed-8555-7b1b04347a91 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:14.598309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67954-3659-49ed-8555-7b1b04347a91 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:14.598833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67954-3659-49ed-8555-7b1b04347a91 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:14.600670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 225/456] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:14.606037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d351-f9cf-431c-9dc3-af9720887209 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:14.606867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d351-f9cf-431c-9dc3-af9720887209 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:14.607352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d351-f9cf-431c-9dc3-af9720887209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:14.634008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8d90d351-f9cf-431c-9dc3-af9720887209 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:14.635546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8d90d351-f9cf-431c-9dc3-af9720887209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:14.663704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:14.738471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 232/457] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:14.779107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67954-3659-49ed-8555-7b1b04347a91 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:14.790879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b83b1-2781-4e40-8f96-32b743e223be None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:14.791505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b83b1-2781-4e40-8f96-32b743e223be None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:14.795058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b83b1-2781-4e40-8f96-32b743e223be None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:14.797793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 233/458] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:14 2022] GET /identity => generated 271 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:14.801907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f074-2092-43cb-97a2-1721d4ddaae0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:14.802569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f074-2092-43cb-97a2-1721d4ddaae0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:14.803223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f074-2092-43cb-97a2-1721d4ddaae0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:14.805477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 234/459] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:14.806804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69f2b-0543-406d-a1ec-fcab40d417e2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:14.807319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69f2b-0543-406d-a1ec-fcab40d417e2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:14.807806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69f2b-0543-406d-a1ec-fcab40d417e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:14.827541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d7d69f2b-0543-406d-a1ec-fcab40d417e2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:14.829120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d7d69f2b-0543-406d-a1ec-fcab40d417e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:14.916658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67954-3659-49ed-8555-7b1b04347a91 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:14.925948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 235/460] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:14.927544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff47383-e752-4693-89dd-1d98eb3e8a90 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:14.928034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff47383-e752-4693-89dd-1d98eb3e8a90 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:14.928576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff47383-e752-4693-89dd-1d98eb3e8a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:14.947941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-aff47383-e752-4693-89dd-1d98eb3e8a90 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:14.949527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-aff47383-e752-4693-89dd-1d98eb3e8a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:14.971871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:14.972572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:14.972919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:14.973227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:14.984937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:14.987652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:15.005348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:15.006268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:15.007146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:15.007146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:15.007146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:15.007146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:15.009551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 226/461] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:15.010647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.030038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 236/462] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:15.030993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff47383-e752-4693-89dd-1d98eb3e8a90 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.048361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:15.057357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:15.127312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3225522-0d20-4d1f-b374-5cefc39a0bb8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:15.153956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff47383-e752-4693-89dd-1d98eb3e8a90 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:15.174461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:15.175410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.175877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.176365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.188541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:15.191696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.192667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:15.194051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.194589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.195491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.196701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:15.197321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:15.198896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:15.209366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:15.210374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:15.211225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:15.211225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:15.211225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:15.211225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:15.213878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 227/463] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:15.215048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 237/464] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:15.219967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.220123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.332921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:15.340521 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:15.353104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0d74d-83b8-4eb7-8a27-3d45e53b6117 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:15.360509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754fa3d-e661-49c4-b167-b1ffba7c89b9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:15.399922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c3ee1-fee0-48e3-bc47-437746b5c5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:15.400723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c3ee1-fee0-48e3-bc47-437746b5c5bb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.401167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c3ee1-fee0-48e3-bc47-437746b5c5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.401440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c3ee1-fee0-48e3-bc47-437746b5c5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.410098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:15.410825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.411645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.412028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.413361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:15.413749 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:15.414990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c3ee1-fee0-48e3-bc47-437746b5c5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:15.415474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.416448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c3ee1-fee0-48e3-bc47-437746b5c5bb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.433647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 238/465] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:15.435021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-dc0c3ee1-fee0-48e3-bc47-437746b5c5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:15.437631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:15.438434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:15.439311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:15.439311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:15.439311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:15.439311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:15.440290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 228/466] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:15.441545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da459a-27da-4c76-bfc0-fa0fa46fb83a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.442308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da459a-27da-4c76-bfc0-fa0fa46fb83a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.442937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da459a-27da-4c76-bfc0-fa0fa46fb83a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.445268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 229/467] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:15.445268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da459a-27da-4c76-bfc0-fa0fa46fb83a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.560947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83915cfe-ab7e-4713-a1b2-b147b8a9157f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:15.578716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da459a-27da-4c76-bfc0-fa0fa46fb83a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:15.597369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:15.597913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.598192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.598536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.599314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:15.601094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.609019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:15.619827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dd43e-6db3-4dec-85b3-3ca93f98d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:15.623896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dd43e-6db3-4dec-85b3-3ca93f98d380 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.623896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dd43e-6db3-4dec-85b3-3ca93f98d380 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.623896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dd43e-6db3-4dec-85b3-3ca93f98d380 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.623896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310dd43e-6db3-4dec-85b3-3ca93f98d380 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:15.623896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310dd43e-6db3-4dec-85b3-3ca93f98d380 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.709786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 239/468] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:16:15 2022] POST /identity/v3/users => generated 247 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:15.710646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.745194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:15.754062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 230/469] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:16:15 2022] POST /identity/v3/users => generated 249 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:15.755761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12519096-ae03-4c82-a53a-b679ff6776f0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.756609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12519096-ae03-4c82-a53a-b679ff6776f0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.757127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12519096-ae03-4c82-a53a-b679ff6776f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.784088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-12519096-ae03-4c82-a53a-b679ff6776f0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:15.786683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-12519096-ae03-4c82-a53a-b679ff6776f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:15.842489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19385196-5d82-4de0-95eb-81c2f6b67d9b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:15.888264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:15.888997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:15.889355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:15.889698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:15.890494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:15.890886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:15.896499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:15.908698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 231/470] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:15.919203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 240/471] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:15.925944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12519096-ae03-4c82-a53a-b679ff6776f0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:15.976420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:16.044450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:16.056954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12519096-ae03-4c82-a53a-b679ff6776f0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:16.092321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:16.107320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0980f8f-f82a-45f5-8b85-8cc14d1ea2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:16.107949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0980f8f-f82a-45f5-8b85-8cc14d1ea2b6 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.108230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0980f8f-f82a-45f5-8b85-8cc14d1ea2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.108527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0980f8f-f82a-45f5-8b85-8cc14d1ea2b6 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.109238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0980f8f-f82a-45f5-8b85-8cc14d1ea2b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:16.111294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0980f8f-f82a-45f5-8b85-8cc14d1ea2b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:16.112955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cf530-aec4-4add-ba01-db4436e3d17d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:16.154288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c0e03-8dd7-42ae-acb0-a65c73e843f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:16.155060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c0e03-8dd7-42ae-acb0-a65c73e843f0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.155360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c0e03-8dd7-42ae-acb0-a65c73e843f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.155656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c0e03-8dd7-42ae-acb0-a65c73e843f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.169092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c0e03-8dd7-42ae-acb0-a65c73e843f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:16.172503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c0e03-8dd7-42ae-acb0-a65c73e843f0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-306c0e03-8dd7-42ae-acb0-a65c73e843f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:16.188312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:16.190123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:16.190985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:16.190985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:16.190985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:16.190985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:16.190985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 241/472] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:16.192135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b773c-a9fe-4ebb-b02a-6546b8750754 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.192425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b773c-a9fe-4ebb-b02a-6546b8750754 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.192733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b773c-a9fe-4ebb-b02a-6546b8750754 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.198041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 242/473] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:16 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:16.200226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b773c-a9fe-4ebb-b02a-6546b8750754 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:16.225039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 232/474] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:16:15 2022] POST /identity/v3/users => generated 249 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:16.226302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e7a6a-3ad0-42e8-9d3c-b5052b81de71 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.226597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e7a6a-3ad0-42e8-9d3c-b5052b81de71 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.226883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e7a6a-3ad0-42e8-9d3c-b5052b81de71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.281630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-550e7a6a-3ad0-42e8-9d3c-b5052b81de71 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:16.283634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-550e7a6a-3ad0-42e8-9d3c-b5052b81de71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:16.349164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b773c-a9fe-4ebb-b02a-6546b8750754 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:16.351062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:16.395204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:16.396011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.396302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.396734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.397448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:16.397754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:16.399245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:16.417335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 243/475] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:16.424227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:16.464051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 233/476] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:16.495204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e7a6a-3ad0-42e8-9d3c-b5052b81de71 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:16.577236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac24c-b580-401c-a50f-781901af22d7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:16.619614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:16.621074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.621500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.621944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.622834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:16.624576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:16.628048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e7a6a-3ad0-42e8-9d3c-b5052b81de71 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:16.647983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:16.679370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1281-5544-411b-bc63-544194719352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:16.679943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1281-5544-411b-bc63-544194719352 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.680230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1281-5544-411b-bc63-544194719352 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.680552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1281-5544-411b-bc63-544194719352 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.681292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c1281-5544-411b-bc63-544194719352 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:16.682669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c1281-5544-411b-bc63-544194719352 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:16.717426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 244/477] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:16:16 2022] POST /identity/v3/users => generated 250 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:16.745837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.sql.core [None req-c85c1281-5544-411b-bc63-544194719352 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 15:16:16.745837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 15:16:16.745837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [parameters: {'id': '7357d7f5d7b0461f83abe53f095f0ec7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 15:16:16.745837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80378) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c85c1281-5544-411b-bc63-544194719352 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Jul 26 15:16:16.770255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 15:16:16.771186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application yield resource Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise self.value Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application session.commit() Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 15:16:16.772204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.flush() Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application compat.raise_( Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 15:16:16.773098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 15:16:16.773984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 15:16:16.774796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [parameters: {'id': '7357d7f5d7b0461f83abe53f095f0ec7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:16.775582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 15:16:16.776371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:16.777718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 234/478] 10.222.0.128 () {60 vars in 1136 bytes} [Tue Jul 26 15:16:16 2022] POST /identity/v3/roles => generated 144 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:16.784762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:16.824063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:16.926231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95515aa-3902-4518-b886-cb8bffb40c7a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:16.962835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dbe3c-563a-45ee-b7a9-78463067ea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:16.963718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbe3c-563a-45ee-b7a9-78463067ea26 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:16.964165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbe3c-563a-45ee-b7a9-78463067ea26 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:16.964613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbe3c-563a-45ee-b7a9-78463067ea26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:16.976683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dbe3c-563a-45ee-b7a9-78463067ea26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d72dbe3c-563a-45ee-b7a9-78463067ea26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:16.983933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:16.985065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 15:16:16.986513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:16.988965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 245/479] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:16.997593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1281-5544-411b-bc63-544194719352 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:17.124736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1281-5544-411b-bc63-544194719352 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:17.168894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf8453-61a8-4a9d-87ab-9ce7377f9cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:17.169528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf8453-61a8-4a9d-87ab-9ce7377f9cbd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:17.169807 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf8453-61a8-4a9d-87ab-9ce7377f9cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:17.170078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf8453-61a8-4a9d-87ab-9ce7377f9cbd None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:17.170798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf8453-61a8-4a9d-87ab-9ce7377f9cbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:17.171102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf8453-61a8-4a9d-87ab-9ce7377f9cbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:17.172526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf8453-61a8-4a9d-87ab-9ce7377f9cbd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:17.183902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 235/480] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:16 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:18.198827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795b24e-73c5-4bf7-bf53-f148a23cd842 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.199391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795b24e-73c5-4bf7-bf53-f148a23cd842 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.199466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795b24e-73c5-4bf7-bf53-f148a23cd842 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.201497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 246/481] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:18.207439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321be0ed-c7de-4e8e-af41-c628aa69c6e0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.208116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321be0ed-c7de-4e8e-af41-c628aa69c6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.208116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321be0ed-c7de-4e8e-af41-c628aa69c6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.208954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81fb99-7dd0-48be-9562-28719c394635 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.209628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81fb99-7dd0-48be-9562-28719c394635 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.213856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81fb99-7dd0-48be-9562-28719c394635 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.213856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 247/482] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:18 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:18.218354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765cc4a-4d9c-4b60-adc5-d024be1182d2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.218354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765cc4a-4d9c-4b60-adc5-d024be1182d2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.218560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765cc4a-4d9c-4b60-adc5-d024be1182d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.229319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-321be0ed-c7de-4e8e-af41-c628aa69c6e0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:18.230580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-321be0ed-c7de-4e8e-af41-c628aa69c6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:18.257641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6765cc4a-4d9c-4b60-adc5-d024be1182d2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:18.258850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6765cc4a-4d9c-4b60-adc5-d024be1182d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:18.288837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:18.372271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 236/483] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:18.397459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321be0ed-c7de-4e8e-af41-c628aa69c6e0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:18.429798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 248/484] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:16:18.460053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765cc4a-4d9c-4b60-adc5-d024be1182d2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:18.483072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:18.588187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321be0ed-c7de-4e8e-af41-c628aa69c6e0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:18.632985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:18.633650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.633933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.634210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.655965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:18.660933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:18.661821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:18.662619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:18.666742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 237/485] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:18.670311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765cc4a-4d9c-4b60-adc5-d024be1182d2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:18.672240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:18.736265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:18.736868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.737149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.737424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.738761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:18.763903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:18.770118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:18.772875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:18.774821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 249/486] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:18.780502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:18.870876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91799dfa-80b9-48dc-a9ec-ad38e660d59d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:18.877529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:18.916358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253b18f-b0f6-4f13-b9c6-ebcac61ceab3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:18.922316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a291f-efc3-411d-829c-31e26b706c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:18.923772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a291f-efc3-411d-829c-31e26b706c77 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.924195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a291f-efc3-411d-829c-31e26b706c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.924497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a291f-efc3-411d-829c-31e26b706c77 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.925228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a291f-efc3-411d-829c-31e26b706c77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:18.925952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a291f-efc3-411d-829c-31e26b706c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:18.927691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a291f-efc3-411d-829c-31e26b706c77 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:18.940620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 238/487] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:18 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:18.941954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bd50b-f2df-4bad-9bc3-fb12cd54aac8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.942248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bd50b-f2df-4bad-9bc3-fb12cd54aac8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.942526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bd50b-f2df-4bad-9bc3-fb12cd54aac8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.944025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 239/488] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:18.949338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bd50b-f2df-4bad-9bc3-fb12cd54aac8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:18.963334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c35942-2355-4aa1-bba7-d09f4734296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:18.963963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35942-2355-4aa1-bba7-d09f4734296a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.964247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35942-2355-4aa1-bba7-d09f4734296a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.964542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35942-2355-4aa1-bba7-d09f4734296a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:18.965269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c35942-2355-4aa1-bba7-d09f4734296a None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:18.965579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c35942-2355-4aa1-bba7-d09f4734296a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:18.966972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c35942-2355-4aa1-bba7-d09f4734296a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:18.976251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 250/489] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:18 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:18.977562 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36e665-2c1c-4400-ba62-6c8d4d47137f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:18.977854 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36e665-2c1c-4400-ba62-6c8d4d47137f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:18.978131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36e665-2c1c-4400-ba62-6c8d4d47137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.002843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fc36e665-2c1c-4400-ba62-6c8d4d47137f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:19.004501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fc36e665-2c1c-4400-ba62-6c8d4d47137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:19.077896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:19.091280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bd50b-f2df-4bad-9bc3-fb12cd54aac8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.098683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 251/490] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:19.099760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36e665-2c1c-4400-ba62-6c8d4d47137f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.137292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.137954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.138507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.138791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.154437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:19.156895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:19.179382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:19.180331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:19.181208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:19.181208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:19.181208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:19.181208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:19.181721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 240/491] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:19.182415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.197378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:19.249778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36e665-2c1c-4400-ba62-6c8d4d47137f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.289398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.290012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.290298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.290577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.301750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:19.303212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:19.319532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:19.320644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:19.321608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:19.321608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:19.321608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:19.321608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:19.321608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 252/492] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:19.322540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24544f42-87e1-48c7-a3c4-e612bb551ba7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.326221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.370317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.371651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.372104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.372567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.388687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:19.397423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:19.398653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:19.399554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:19.407757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 241/493] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:19.407757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.413282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:19.453789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eafd65-9487-4b76-80b2-e3a1458b39e4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.474406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:19.505180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.505559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.505751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.506035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.507701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:19.508037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:19.509486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:19.527882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 253/494] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:19.528877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.539525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf28e7a-2a97-4977-b3b8-447abaad7e60 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.586818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.587906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.588652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.589116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.590004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:19.590493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:19.592599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:19.606475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 242/495] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:19.607550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.647127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf1d9-71ac-48b1-9a25-61e385a33230 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.680154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:19.682716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfe820-4e92-4941-b985-be2c16936cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.683338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfe820-4e92-4941-b985-be2c16936cef None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.683624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfe820-4e92-4941-b985-be2c16936cef None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.683905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfe820-4e92-4941-b985-be2c16936cef None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.684659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfe820-4e92-4941-b985-be2c16936cef None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:19.685384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfe820-4e92-4941-b985-be2c16936cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:19.686445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfe820-4e92-4941-b985-be2c16936cef None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:19.700369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 254/496] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:19.701667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb94801-9105-4cb3-b196-4945d6753df1 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.701945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb94801-9105-4cb3-b196-4945d6753df1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.702224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb94801-9105-4cb3-b196-4945d6753df1 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.708626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 255/497] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:19.708626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb94801-9105-4cb3-b196-4945d6753df1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.729199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76891d5b-7a32-404c-82d4-12f320d971ce None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.745973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:19.774364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.775037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.775326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.775599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.783524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:19.785030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:19.806121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:19.807053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 243/498] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:19.812953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:19.848600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb94801-9105-4cb3-b196-4945d6753df1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.898226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa5429-51df-4244-8ef7-660b01cf6ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:19.898882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa5429-51df-4244-8ef7-660b01cf6ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.899167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa5429-51df-4244-8ef7-660b01cf6ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.899446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa5429-51df-4244-8ef7-660b01cf6ee5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.913356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa5429-51df-4244-8ef7-660b01cf6ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:19.914842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa5429-51df-4244-8ef7-660b01cf6ee5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6aaa5429-51df-4244-8ef7-660b01cf6ee5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:19.930303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:19.931378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:19.932283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 256/499] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:19.933574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1f752-fce7-4c99-81a9-0f0dcf1be015 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:19.933675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1f752-fce7-4c99-81a9-0f0dcf1be015 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:19.934083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1f752-fce7-4c99-81a9-0f0dcf1be015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:19.953764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-83f1f752-fce7-4c99-81a9-0f0dcf1be015 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:19.955448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-83f1f752-fce7-4c99-81a9-0f0dcf1be015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:19.960432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8a4ab-d794-4413-a59d-05b4b3a205bb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:19.991070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.005210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.005862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.006143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.009544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.019297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:20.020942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:20.037405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:20.038526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:20.039421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:20.039421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:20.039421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:20.039421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:20.039421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 244/500] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:20.039421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.060123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 257/501] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:20.060123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1f752-fce7-4c99-81a9-0f0dcf1be015 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.083375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.177750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603e704-612f-4470-be11-04ac965c3997 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.179514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1f752-fce7-4c99-81a9-0f0dcf1be015 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.216224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.216542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.216882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.217164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.217278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.217438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.217525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.217718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.218243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:20.218519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:20.218606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:20.218833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:20.220844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:20.221056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:20.261041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 258/502] 10.222.0.128 () {56 vars in 1142 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:20.262086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 245/503] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:20.262606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.263622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.307387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.359845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.379934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4b3b2-bb02-46fc-914e-40a6ca900669 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.386978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1aa99-8c0f-4f69-9532-d63b44887eb1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.424489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda2e69-206a-4344-acdb-6f8ba6e40140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.425125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2e69-206a-4344-acdb-6f8ba6e40140 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.425590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2e69-206a-4344-acdb-6f8ba6e40140 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.426035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2e69-206a-4344-acdb-6f8ba6e40140 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.427054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.427383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda2e69-206a-4344-acdb-6f8ba6e40140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:20.427707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.427927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda2e69-206a-4344-acdb-6f8ba6e40140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:20.428025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.428257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.430482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda2e69-206a-4344-acdb-6f8ba6e40140 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:20.439609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:20.443946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 259/504] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:20.444656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:20.445670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:20.446681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:20.447485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc984d-feb4-4285-893b-29ebfcd867a2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.447485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc984d-feb4-4285-893b-29ebfcd867a2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.447485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc984d-feb4-4285-893b-29ebfcd867a2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.447664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 246/505] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:20.447664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.447781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 260/506] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:20.447931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc984d-feb4-4285-893b-29ebfcd867a2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.572275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.577007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc984d-feb4-4285-893b-29ebfcd867a2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.582672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30547c54-0d21-4bd3-9294-3f6ebc729864 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.622304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.622843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.623122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.623398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.623486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.624427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.624978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.625474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.638641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.714686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:20.716173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:20.716267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:20.717744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:20.824556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 261/507] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:20.825414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.828001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.836532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 247/508] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:20.842969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:20.880135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:20.948365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e4eba-08b5-411b-a66b-ee7d4b8ee766 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.950625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa48430-5f3e-49fe-bdc7-9f913118952a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:20.987771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c27f8c-00aa-4861-af29-ba1af53d9d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.988405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27f8c-00aa-4861-af29-ba1af53d9d4a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.988717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27f8c-00aa-4861-af29-ba1af53d9d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.988994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27f8c-00aa-4861-af29-ba1af53d9d4a None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.989347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:20.989718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c27f8c-00aa-4861-af29-ba1af53d9d4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:20.990027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c27f8c-00aa-4861-af29-ba1af53d9d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:20.990402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:20.990753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:20.991033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:20.991418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c27f8c-00aa-4861-af29-ba1af53d9d4a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.001825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.003611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 248/509] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:21.003993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.005229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3710d-c393-49d4-a0fe-f10490f6c4ba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.005529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3710d-c393-49d4-a0fe-f10490f6c4ba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.005806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3710d-c393-49d4-a0fe-f10490f6c4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:21.020079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:21.021222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 262/510] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:21.022099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.028976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-56a3710d-c393-49d4-a0fe-f10490f6c4ba None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:21.030200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-56a3710d-c393-49d4-a0fe-f10490f6c4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:21.088108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.106089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 249/511] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:21.106923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3710d-c393-49d4-a0fe-f10490f6c4ba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.145740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f44e91-5de1-4324-be70-08086c499aca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.147659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.182599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.183751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.184284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.184681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.197343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.198841 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:21.213810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:21.215474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:21.217116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:21.217116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:21.217116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.217116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:21.217116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 263/512] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:21.217116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.227252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3710d-c393-49d4-a0fe-f10490f6c4ba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.264665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.265321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.265605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.265881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.277457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.279058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:21.294386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:21.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:21.298660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:21.298660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:21.298660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.298660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:21.298660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 250/513] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:21.298660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.342149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b634cc-d890-42b9-8b3b-3314a520f11f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.349268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.383325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.384017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.384306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.384625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.396310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.397768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.412577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3da64-693c-417e-8112-6b5964d56722 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:21.413111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:21.413906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:21.414761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:21.414761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:21.414761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.414761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:21.414761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 264/514] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:21.415517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.426002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.460373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.460373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.460373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.460373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.460792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:21.460792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.462160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.482800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 251/515] 10.222.0.128 () {56 vars in 1142 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:21.483736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.531397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99731ad4-1699-4f5b-b00f-d4f9f8eecad1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.566223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.566857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.567143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.567414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.577660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.579208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.584161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.589904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a42f3-0954-43b9-a096-a94cab53778a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:21.593976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:21.599198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:21.600038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:21.600038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:21.600038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:21.600038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:21.600038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 265/516] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:21.600038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.628679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.629333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.629619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.629901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.630600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:21.630911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.632378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.644200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 252/517] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:21.645064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.667371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.714319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177f8a8-6a8b-4acd-b08d-35f22cf73b96 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.747497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a76838-774a-4024-bd7e-3e3e2b073266 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.749306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.749930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.750216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.750493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.751284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:21.751550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.752963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.764533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 266/518] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:21.765331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.784553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.785135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.785426 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.785704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.786405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:21.786716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.788088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.800439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 253/519] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:21.801271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.818518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.870335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98831f1-6488-4683-b97a-5c4ac205ac06 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.889484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:21.905535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15237eb-aad4-4c52-a69f-b5b57a0c6456 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:21.906857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.907710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.908002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.908279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.946190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:21.946648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:21.946937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:21.947216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:21.947927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:21.948237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.949664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:21.963236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 254/520] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:21.964032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:21.982768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:21.984270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.044264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:22.068054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca79fde-b319-40bc-82a7-05606720fe8d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.086646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 267/521] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:22.087490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.105224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.105860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.106152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.106428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.107078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:22.113966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.115456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:22.129266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:22.130494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 255/522] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:22.131527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.198744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c984782-522e-400f-984e-bc172487ba87 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.234693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.235360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.235779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.236056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.236146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7ba72-de80-4521-915f-aa9411cd5f79 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.243015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.244493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:22.257240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:22.258127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 268/523] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:22.259653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.273747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.274369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.274653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.274931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.277778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:22.283074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.284579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:22.295720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:22.308985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 256/524] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:22.309882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.339530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:22.360680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4e800-41d5-4bed-b9ec-dfacd2722f81 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.397936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.398545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.398826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.399098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.399375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41f1e1-535a-4dd0-a5f7-1ed7c203851e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.399777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.401259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.432712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.433356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.433642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.433923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.445396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.446854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:22.462964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:22.464422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:22.465613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:22.465613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:22.465613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:22.465613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:22.465832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 269/525] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:16:22 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:22.465911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 257/526] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:22.470071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.470532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.538275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:22.605259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79203ebf-015b-4801-807b-c696acce3335 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.613894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926593b-2755-470f-895d-3ce8d5e3c636 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.647245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.648253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.648253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.648548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.649323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:22.649638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.651095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.665874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.666655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.668135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.668417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.669771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:22.669771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.671001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.674242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:22.676141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 258/527] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:22.677328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.700119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 270/528] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:22.701619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.810736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda4b4a-cca0-4c1c-955a-c6ce3dfab9b3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.822142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:22.827098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53697255-4731-4559-81b3-203e59c05e44 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:22.852073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.852738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.853027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.853310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.854175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:22.854494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.855959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.866271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:22.866843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:22.867127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:22.867669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:22.868378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:22.868721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:22.870124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:22.881737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 259/529] 10.222.0.128 () {56 vars in 1144 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:22.882604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.888314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 271/530] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:22.889241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:22.953671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:23.001368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba890-05bb-4eed-b17b-72909937b3e0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:23.001562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e2321-f150-4904-b65a-42b1025f537e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:23.042089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:23.042721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:23.043002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:23.043281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:23.044080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=92e40a33db1a41d59d9a7a3bef5f3ece) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:23.044391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:23.046087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:23.046648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:23.047288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:23.048155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:23.048436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:23.049271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=0207287721f5489f8ac7a258356e57bd) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:23.049589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:23.051031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:23.052981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 272/531] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/role_assignments?user.id=92e40a33db1a41d59d9a7a3bef5f3ece&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:23.053690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:23.068531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 260/532] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:22 2022] GET /identity/v3/role_assignments?user.id=0207287721f5489f8ac7a258356e57bd&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:23.069777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:23.117964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:23.218699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b13577-19e9-478d-b1f3-edfa9bd29893 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:23.263779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737123c-fa59-4366-9d91-c06e6ec3c881 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:23.264081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:23.264762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:23.265054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:23.265325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:23.274261 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:23.275861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:23.289670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:23.290545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:23.291375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 261/533] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:23.297110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:23.317918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:23.360597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:23.361148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:23.361870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:23.361870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:23.412729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:23.429871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133aa4-1339-4002-9593-bb3f42fffdb4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:23.493554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:23.495383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:23.517455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:23.518172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:23.518458 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:23.518737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:23.519501 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:23.519820 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:23.521420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:23.554834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 262/534] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:23.556268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:23.651703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 273/535] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:23.657508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:23.708166 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441237e-3b67-401a-b712-c5b73fc92aea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:23.709973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:23.757062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:23.757693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:23.757980 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:23.758260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:23.759064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=8ff6f527aa364534b2fdd0b770924e83) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:23.759382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:23.760994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:23.767700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 263/536] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:23 2022] GET /identity/v3/role_assignments?user.id=8ff6f527aa364534b2fdd0b770924e83&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:23.837205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8a0ac-ab72-4e0b-9ac5-71c9079a0928 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:23.876610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:23.877328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:23.877618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:23.877898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:23.888952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:23.890446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:23.906514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:23.908128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:23.909144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:23.909144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:23.909144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:23.909144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:23.909144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 274/537] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:23.917801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:23.952871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:24.061843 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928231a-9ee0-4b3e-ad1d-7c5e885e1113 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:24.103853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:24.104495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:24.104785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:24.105065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:24.105766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:24.106083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:24.107445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:24.125168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 264/538] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:24.131033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:24.203463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:24.236292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845948d-93a0-4ef8-ad15-3100aa7523cf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:24.273041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:24.273802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:24.274085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:24.274364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:24.281545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:24.283759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:24.296591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:24.298156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:24.299172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 275/539] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:24.309636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:24.411051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:24.442272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45eaf53-0fd2-4a20-8ea2-21b204e22914 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:24.505250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651f4bd-df8b-4ae6-9247-b0b89e7d2772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:24.505901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651f4bd-df8b-4ae6-9247-b0b89e7d2772 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:24.506186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651f4bd-df8b-4ae6-9247-b0b89e7d2772 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:24.506461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651f4bd-df8b-4ae6-9247-b0b89e7d2772 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:24.507232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651f4bd-df8b-4ae6-9247-b0b89e7d2772 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:24.507546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651f4bd-df8b-4ae6-9247-b0b89e7d2772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:24.509040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651f4bd-df8b-4ae6-9247-b0b89e7d2772 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:24.523885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 265/540] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:24.529850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:24.648244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf1172-99c0-464b-a980-34db87f4f8eb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:24.683529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:24.690191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011ed44-e795-4676-ab9a-1d77c9aa1670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:24.690818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011ed44-e795-4676-ab9a-1d77c9aa1670 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:24.691105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011ed44-e795-4676-ab9a-1d77c9aa1670 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:24.691382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011ed44-e795-4676-ab9a-1d77c9aa1670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:24.692238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011ed44-e795-4676-ab9a-1d77c9aa1670 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=c6ecc741022f4d6189f4d3f19b4cdbe0) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:24.692869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011ed44-e795-4676-ab9a-1d77c9aa1670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:24.702858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011ed44-e795-4676-ab9a-1d77c9aa1670 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:24.702858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 276/541] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:24 2022] GET /identity/v3/role_assignments?user.id=c6ecc741022f4d6189f4d3f19b4cdbe0&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:25.040686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d601-901f-472b-9d0e-b45f9a558515 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.041068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d601-901f-472b-9d0e-b45f9a558515 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.041269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d601-901f-472b-9d0e-b45f9a558515 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.042895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 266/542] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:25.050404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190cab2-9276-41d9-8cc1-fcff3ebc382b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.050992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190cab2-9276-41d9-8cc1-fcff3ebc382b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.051473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190cab2-9276-41d9-8cc1-fcff3ebc382b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.082266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f190cab2-9276-41d9-8cc1-fcff3ebc382b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:25.083503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f190cab2-9276-41d9-8cc1-fcff3ebc382b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:25.177077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 277/543] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:25.208870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043d601-901f-472b-9d0e-b45f9a558515 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:25.363239 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043d601-901f-472b-9d0e-b45f9a558515 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:25.407893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b3a0d-5164-45ea-b7be-5cc04137ab6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:25.408594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b3a0d-5164-45ea-b7be-5cc04137ab6e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.408880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b3a0d-5164-45ea-b7be-5cc04137ab6e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.409158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b3a0d-5164-45ea-b7be-5cc04137ab6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.412354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:25.427090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b3a0d-5164-45ea-b7be-5cc04137ab6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-965b3a0d-5164-45ea-b7be-5cc04137ab6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:25.432106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:25.441240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:25.442052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 267/544] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:25.442714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190cab2-9276-41d9-8cc1-fcff3ebc382b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:25.566357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0644f74-ade9-4ae9-b786-1ab4eb32e038 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.566884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0644f74-ade9-4ae9-b786-1ab4eb32e038 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.566984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0644f74-ade9-4ae9-b786-1ab4eb32e038 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.568249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 268/545] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:25.573765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a53fd-72c9-4676-88ee-bd03d0838d29 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.574053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a53fd-72c9-4676-88ee-bd03d0838d29 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.574333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a53fd-72c9-4676-88ee-bd03d0838d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.586825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190cab2-9276-41d9-8cc1-fcff3ebc382b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:25.612885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:25.616495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3a2a53fd-72c9-4676-88ee-bd03d0838d29 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:25.617946 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3a2a53fd-72c9-4676-88ee-bd03d0838d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:25.632680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:25.633578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.634700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.634700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.635400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:25.635847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:25.637400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:25.653539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 278/546] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:25 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:25.661473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:25.720727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 269/547] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:16:25.743383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a53fd-72c9-4676-88ee-bd03d0838d29 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:25.853301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a53fd-72c9-4676-88ee-bd03d0838d29 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:25.878346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d671-00c8-4a69-9b36-efc119af5c2a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:25.880034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:25.890826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f978d99-867c-449c-bb2b-27134319b127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:25.892841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f978d99-867c-449c-bb2b-27134319b127 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.893190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f978d99-867c-449c-bb2b-27134319b127 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.893531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f978d99-867c-449c-bb2b-27134319b127 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.904531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f978d99-867c-449c-bb2b-27134319b127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5f978d99-867c-449c-bb2b-27134319b127 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:25.910750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:25.911679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:25.912507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 270/548] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:25.919504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a28ed5-3bd6-4445-920c-7cee2803465a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.920159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a28ed5-3bd6-4445-920c-7cee2803465a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.921123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a28ed5-3bd6-4445-920c-7cee2803465a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.923161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 271/549] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:25 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:25.927659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a28ed5-3bd6-4445-920c-7cee2803465a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:25.928995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a652f-0dd2-4bbd-9727-47ec9f1e700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:25.930557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a652f-0dd2-4bbd-9727-47ec9f1e700d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.930848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a652f-0dd2-4bbd-9727-47ec9f1e700d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.931122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a652f-0dd2-4bbd-9727-47ec9f1e700d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.945890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a652f-0dd2-4bbd-9727-47ec9f1e700d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:25.947735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a652f-0dd2-4bbd-9727-47ec9f1e700d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d44a652f-0dd2-4bbd-9727-47ec9f1e700d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:25.971751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:25.972694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:25.973555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:25.973555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:25.973555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:25.973555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:25.973555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 279/550] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:25.974595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de476-3a4f-4d62-9760-17435ec3e769 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.974883 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de476-3a4f-4d62-9760-17435ec3e769 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.975160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de476-3a4f-4d62-9760-17435ec3e769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.001342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b03de476-3a4f-4d62-9760-17435ec3e769 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:26.002526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b03de476-3a4f-4d62-9760-17435ec3e769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:26.038257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:26.047467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a28ed5-3bd6-4445-920c-7cee2803465a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.084270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.084923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.085204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.085477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.086194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:26.086594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.087016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 280/551] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:26.087859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.088273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de476-3a4f-4d62-9760-17435ec3e769 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.111981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 272/552] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:25 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:26.119324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.181586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:26.258238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de476-3a4f-4d62-9760-17435ec3e769 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.270884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e6907-c732-44d5-bace-e2317c76a3e1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.322844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.324393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.324711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.325586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.326305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:26.327279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.327924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.328251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.329143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.329456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.336485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.340312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:26.347329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:26.348257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:26.349122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:26.349743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 281/553] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:26.350386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.355672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 273/554] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:26.355672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.394959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:26.484483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf346c-a20d-4b62-a25f-f1cf164dc206 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.486930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985db4-462e-46b9-b966-a3e2e20df9ca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.496253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:26.524538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095f807-b53e-410b-b054-0abfb5dfbd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.525519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095f807-b53e-410b-b054-0abfb5dfbd9b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.525812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095f807-b53e-410b-b054-0abfb5dfbd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.526243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095f807-b53e-410b-b054-0abfb5dfbd9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.533297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.533864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.534154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.534436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.535166 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:26.535478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.537363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.540738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095f807-b53e-410b-b054-0abfb5dfbd9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.542193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095f807-b53e-410b-b054-0abfb5dfbd9b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.552223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 274/555] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:26.553185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a095f807-b53e-410b-b054-0abfb5dfbd9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:26.561674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:26.562594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:26.563548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:26.563548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:26.563548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:26.563548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:26.570313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 282/556] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:26.571905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2526-1f7f-47a2-bf81-bbe7bdb2bfa7 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.572393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2526-1f7f-47a2-bf81-bbe7bdb2bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.572906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2526-1f7f-47a2-bf81-bbe7bdb2bfa7 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.574611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 283/557] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:26.575558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d2526-1f7f-47a2-bf81-bbe7bdb2bfa7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.682646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:26.684994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d3f2-b248-4940-9a6d-1401450e8897 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.706563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d2526-1f7f-47a2-bf81-bbe7bdb2bfa7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.725211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.725788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.726077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.726358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.734207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.735959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.745998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503d2f7-ec88-4185-888f-e948b2f3a9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.746643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503d2f7-ec88-4185-888f-e948b2f3a9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.746924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503d2f7-ec88-4185-888f-e948b2f3a9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.747197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503d2f7-ec88-4185-888f-e948b2f3a9e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:26.747746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:26.749159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 275/558] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:26.750083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.764266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503d2f7-ec88-4185-888f-e948b2f3a9e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.766717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503d2f7-ec88-4185-888f-e948b2f3a9e9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f503d2f7-ec88-4185-888f-e948b2f3a9e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:26.783646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:26.784618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:26.785515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:26.785515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:26.785515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:26.785515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:26.785515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 284/559] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:26.786304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359edce-0517-491e-b686-a887acb2bf97 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.786304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359edce-0517-491e-b686-a887acb2bf97 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.786631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359edce-0517-491e-b686-a887acb2bf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.787165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:26.805131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5359edce-0517-491e-b686-a887acb2bf97 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:26.806281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5359edce-0517-491e-b686-a887acb2bf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:26.872067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e9f2d-8d0c-4b95-b6f9-d3102c000e40 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.888495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 285/560] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:26.888495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359edce-0517-491e-b686-a887acb2bf97 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.923701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.924360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.924644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.924929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.925636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:26.926150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.927390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.944321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 276/561] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:26.944321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.950726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:26.993735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359edce-0517-491e-b686-a887acb2bf97 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.037062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.037701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.037987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.038266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.039077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:27.039391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.040905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.050756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.059601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 286/562] 10.222.0.128 () {56 vars in 1140 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:27.064354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.071247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00789-0bfb-43a1-8629-53d4c18778b1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.120957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.121679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.121968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.122242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.122938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:27.123243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.124626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.136945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 277/563] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:27.137780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.187588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915c3be-24b2-42ce-9259-5b8b9d8b3432 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.209586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.224644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.225314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.225601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.225882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.237887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:27.243232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:27.244604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:27.245601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 287/564] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:27.246314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.257376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41c582-d647-4a4c-a070-5fb65edb5d3a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.314947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.315617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.315929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.316211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.325033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.326525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:27.339377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:27.340609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 15:16:27.341764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:27.346106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 278/565] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:27.346913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.358254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.383621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4a18a-77c6-4221-880f-9456100beda6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.419992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.420656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.420941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.421216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.450145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727f36-cf78-49b2-92ad-1a6cf69233a7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.470924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.485943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76506c1f-9201-4a68-abb2-6486631ff9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.486581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506c1f-9201-4a68-abb2-6486631ff9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.486864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506c1f-9201-4a68-abb2-6486631ff9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.487140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506c1f-9201-4a68-abb2-6486631ff9b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.490190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.491648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.495108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76506c1f-9201-4a68-abb2-6486631ff9b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.496490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76506c1f-9201-4a68-abb2-6486631ff9b4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-76506c1f-9201-4a68-abb2-6486631ff9b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:27.508727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:27.509671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:27.510666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 279/566] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:27.511469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b7ca9-9261-4225-9c54-af6c104f77f6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.511574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b7ca9-9261-4225-9c54-af6c104f77f6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.511840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b7ca9-9261-4225-9c54-af6c104f77f6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.514945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 280/567] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:27 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:27.514945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b7ca9-9261-4225-9c54-af6c104f77f6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.585802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.597811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 288/568] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:27.598645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.620865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b7ca9-9261-4225-9c54-af6c104f77f6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.658555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.659186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.659469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.659747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.660491 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:27.660809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.662176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.670271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 281/569] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:27.671121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.684154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.705865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7731-c280-4ea3-96bd-62739862ba09 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.742949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068a2a3-c85c-46c7-92a9-fd93cf6fa772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.743572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068a2a3-c85c-46c7-92a9-fd93cf6fa772 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.743858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068a2a3-c85c-46c7-92a9-fd93cf6fa772 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.744140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068a2a3-c85c-46c7-92a9-fd93cf6fa772 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.744986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068a2a3-c85c-46c7-92a9-fd93cf6fa772 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:27.745356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068a2a3-c85c-46c7-92a9-fd93cf6fa772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.746746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068a2a3-c85c-46c7-92a9-fd93cf6fa772 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.767224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 289/570] 10.222.0.128 () {56 vars in 1142 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:27.768484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63b35-5f97-454c-8948-5d670247c255 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.768782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63b35-5f97-454c-8948-5d670247c255 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.769065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63b35-5f97-454c-8948-5d670247c255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.786744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4db63b35-5f97-454c-8948-5d670247c255 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:27.787971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4db63b35-5f97-454c-8948-5d670247c255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:27.788145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111ef91-4b2f-48b0-8ced-82ca1ab25ea2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.817162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.830175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:27.830809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:27.831094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:27.831373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:27.832181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:27.832560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:27.833961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:27.851629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 282/571] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:27.852486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.865727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 290/572] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:27.866631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db63b35-5f97-454c-8948-5d670247c255 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:27.934470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:27.964109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6185a7-385f-4dcc-8a11-42fa7e93b23b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:27.966998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db63b35-5f97-454c-8948-5d670247c255 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.000841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.001412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.001695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.001972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.002224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.002865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.003147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.003425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.012394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.013398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.013891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.014835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:28.028712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:28.029798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:28.030767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:28.030767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:28.030767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.030767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:28.030984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:28.032900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:28.034926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:28.034926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:28.034926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.034926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:28.035455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 283/573] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:28.035455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.035810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 291/574] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:28.036683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.078212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.137457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21863be-08c7-45d0-9a38-ae0b6ab129d0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.139786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036624f3-fa4a-49ff-8af0-8d5c3a720bd9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.158167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.172217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.173155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.173435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.173707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.174470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.175137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.175423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.175701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.184610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.186057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.186422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.188085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:28.202011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:28.203125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:28.209762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:28.209762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:28.209762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.209762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:28.210065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:28.211084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:28.212153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:28.212153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:28.212153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:28.212153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:28.212426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 292/575] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:28.212426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.212593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 284/576] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:28.212593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.347360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.358492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62941a3c-d5a3-40f8-a3f1-19f58979509b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.360661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3884208-1f90-4978-a19c-a7b91db25f7c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.396840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.397066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.397612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.397757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.397899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.398029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.398876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:28.399184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.400580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.408983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:28.413116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 285/577] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:28.413913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:28.414375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:28.415250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:28.416089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 293/578] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:28.416835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.451896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.532894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e6453-7329-494f-855f-f60b0f102dfb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.533845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70fc4a-19ea-4635-a752-552825a5b297 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.567917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.568249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.568900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.569093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.569202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.569453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.569598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.569950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.571039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:28.571498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.573128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:28.573308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.573452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.574848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.586991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 294/579] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:28.587520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 286/580] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:28.588383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.589155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.604449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.684819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.691117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce270134-e14a-4134-80ff-8ab048793c02 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.693855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8f34d-c99c-4801-bb6c-c4243d7080e4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.723703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.724350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.724696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.724973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.727386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.728004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.728292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.728680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.729377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:28.729735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.731105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.731936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.733383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.742557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 295/581] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:28.743359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:28.745956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:28.746864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:28.747798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 287/582] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:28.748316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.850401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0b060-c5fc-4e98-b667-0c88a736e552 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.862141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11dc00-bad8-4453-97db-431b510fe7fc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:28.886986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.887245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.887608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.887893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.888166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.888923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:28.889229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.890654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.899696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 296/583] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:28.900522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:28.900632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:28.901202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:28.901591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:28.901876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:28.910442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:28.911890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:28.917608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:28.923459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:28.924434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 288/584] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:28.925567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.009341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2135c9-caef-42c5-95e0-482bb82e9bff None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.030081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab22ec-8f6f-4ae9-ab64-939f14ca66b5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.043737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.044381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.044702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.044978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.052163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.053672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.060009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.065148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:29.065460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:29.066477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:29.067403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:29.067832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.067832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.067832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.068036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 297/585] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:29.068036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.075052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.076559 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:29.088658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:29.089734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:29.090833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 289/586] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:29.091334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.148409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.179642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab66d3-590a-423a-b2ba-78b9e619e571 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.199135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a261-2efd-4cb1-941f-e99e1394a333 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.216526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.217273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.217662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.218048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.218971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:29.220234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.222396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.233134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.233798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.234048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.234324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.235815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 298/587] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:29.236655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.251187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.252685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:29.268994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:29.270063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:29.271010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:29.271010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:29.271010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:29.271010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:29.271010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 290/588] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:29.271010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.322685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.351439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d160d-2078-4a05-9cb4-f3d6630b9683 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.379826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467d2ac-b7e0-4e10-a9f9-da9bbe67c95c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.386231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.386849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.387136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.387417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.388216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:29.388565 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.390001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.391795 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.408664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 299/589] 10.222.0.128 () {56 vars in 1142 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:29.409424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.418810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.419437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.419721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.419998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.420787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:29.421093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.423828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.436878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 291/590] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:29.437905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.522005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e742-6aec-46ba-8b41-99b5a51233be None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.550456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532abf54-2436-41f4-b665-6bcc72e9c631 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.553206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.556483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.557121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.557403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.557685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.558446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:29.558758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.560247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.572192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 300/591] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:29.573029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.585922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.586544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.586831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.587137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.587918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=97295dc4a04143fe879bef062964e4e0) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:29.588448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.590186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.596853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 292/592] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/role_assignments?user.id=97295dc4a04143fe879bef062964e4e0&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:29.597823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.636541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.688936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49ba4f-048a-4acf-8fad-63cb86828ab0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.718662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4def8ba-99ee-47c5-b22b-02a5bd0fc0c8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.725889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.726938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.727433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.727934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.729379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:29.729933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.732708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.752038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 301/593] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:29.752923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.773270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.773911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.774192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.774466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.786859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.788369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:29.803454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:29.804620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:29.805523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:29.805523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:29.805523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:29.805523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:29.805523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 293/594] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:29.805523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:29.836379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.877123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4152d-4c1d-4a72-a421-5428e1be316e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.893569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:29.916979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.917928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.918217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.918509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/0207287721f5489f8ac7a258356e57bd/roles/224e0c1f1992436bbcf1ba18913cb81b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.929181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1c1c4-5c63-4b6e-ada0-b3c0132e8aa2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:29.936434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=0207287721f5489f8ac7a258356e57bd, role_id=224e0c1f1992436bbcf1ba18913cb81b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.938061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.966146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:29.966818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:29.967088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:29.967367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/92e40a33db1a41d59d9a7a3bef5f3ece/roles/224e0c1f1992436bbcf1ba18913cb81b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:29.983047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=92e40a33db1a41d59d9a7a3bef5f3ece, role_id=224e0c1f1992436bbcf1ba18913cb81b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:29.984737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:29.994659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 302/595] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:29 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/0207287721f5489f8ac7a258356e57bd/roles/224e0c1f1992436bbcf1ba18913cb81b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:29.995527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.064905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 294/596] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:29 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/92e40a33db1a41d59d9a7a3bef5f3ece/roles/224e0c1f1992436bbcf1ba18913cb81b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:30.066525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.145060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88ae13-2358-42d7-895a-a62944537a4e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:30.159679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:30.206288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f839c-cf93-4dc1-9801-9d8f353294f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:30.210811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:30.211911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:30.213465 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:30.214235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:30.231771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:30.233313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:30.247601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:30.248742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 15:16:30.249686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:30.250783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 303/597] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:30.257481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.273047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:30.274565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:30.275066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:30.275530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:30.277003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:30.277547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:30.279778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:30.296660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 295/598] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:30.297589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:30.307580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.408700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d57d0-8108-4155-8e3c-7bc0f7fb3ecb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:30.457512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:30.458155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:30.458437 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:30.458858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:30.460091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:30.463006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:30.464722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:30.468581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff1278-ae49-44df-a531-c36ed09ea79a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:30.499169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 304/599] 10.222.0.128 () {56 vars in 1144 bytes} [Tue Jul 26 15:16:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:30.505322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.550268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:30.585220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:30.586135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:30.586254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:30.588508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:30.603836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:30.605630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:30.626546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:30.627676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:30.628835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:30.637139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 296/600] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:30.640637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.644803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:30.678196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82510d44-5df7-4778-a942-63a80cd6b3c7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:30.746051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:30.746679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:30.747021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:30.747300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:30.768172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:30.769778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:30.800967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:30.801950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:30.802816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:30.802816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:30.802816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:30.802816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:30.802816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 305/601] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:30.815036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.820161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329fb70-dc43-4f7a-87fe-421717f809f2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:30.857002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:30.857627 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:30.857913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:30.858193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:30.859054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:30.859446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:30.861216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:30.880327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 297/602] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:30.881702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:30.960895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:31.007551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d015f-16b4-4632-93e0-1488eab23af1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:31.034084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578c5bd-bf61-47de-b2bf-ee6b5390f0ca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:31.071408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:31.072086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:31.072367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:31.072671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/8ff6f527aa364534b2fdd0b770924e83/roles/224e0c1f1992436bbcf1ba18913cb81b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:31.092803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:31.095205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=8ff6f527aa364534b2fdd0b770924e83, role_id=224e0c1f1992436bbcf1ba18913cb81b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:31.098408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:31.102273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:31.102912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:31.103235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:31.103511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:31.104172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:31.112311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:31.112311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:31.119394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 306/603] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:31.124330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:31.218863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 298/604] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:30 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/8ff6f527aa364534b2fdd0b770924e83/roles/224e0c1f1992436bbcf1ba18913cb81b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:31.238207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a91f-2b53-4201-be6f-dea7812b7704 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:31.277939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:31.278828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:31.278958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:31.279171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:31.288398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:31.289861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:31.303527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:31.304525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:31.306192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 307/605] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:31.310885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:31.471226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea99b4-892a-45e2-a3f7-2be3fbd20692 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:31.479198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:31.510191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cc467-02b2-4a8b-8fc5-45425c990608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:31.510822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cc467-02b2-4a8b-8fc5-45425c990608 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:31.511111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cc467-02b2-4a8b-8fc5-45425c990608 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:31.511388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cc467-02b2-4a8b-8fc5-45425c990608 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:31.512154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cc467-02b2-4a8b-8fc5-45425c990608 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:31.512485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cc467-02b2-4a8b-8fc5-45425c990608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:31.514028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cc467-02b2-4a8b-8fc5-45425c990608 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:31.528045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 299/606] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:31.534842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:31.631684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:31.709003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1a20-14c3-4a5b-9d90-3127ad11ee8b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:31.762750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52973e-9fab-45f3-bf56-24c66ac8e50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:31.763527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52973e-9fab-45f3-bf56-24c66ac8e50b None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:31.763772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52973e-9fab-45f3-bf56-24c66ac8e50b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:31.764064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52973e-9fab-45f3-bf56-24c66ac8e50b None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/c6ecc741022f4d6189f4d3f19b4cdbe0/roles/224e0c1f1992436bbcf1ba18913cb81b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:31.782348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52973e-9fab-45f3-bf56-24c66ac8e50b None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=c6ecc741022f4d6189f4d3f19b4cdbe0, role_id=224e0c1f1992436bbcf1ba18913cb81b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:31.783812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52973e-9fab-45f3-bf56-24c66ac8e50b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:31.873033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 308/607] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:31 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/c6ecc741022f4d6189f4d3f19b4cdbe0/roles/224e0c1f1992436bbcf1ba18913cb81b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:31.888101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47477a08-8c4d-4495-b822-b4839c7d46f7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:31.888403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47477a08-8c4d-4495-b822-b4839c7d46f7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:31.888634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47477a08-8c4d-4495-b822-b4839c7d46f7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:31.890307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 300/608] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:31.897089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b827a0-c9a3-4efb-975d-79c6e3533742 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:31.897393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b827a0-c9a3-4efb-975d-79c6e3533742 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:31.897655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b827a0-c9a3-4efb-975d-79c6e3533742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:31.918144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-85b827a0-c9a3-4efb-975d-79c6e3533742 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:31.919422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-85b827a0-c9a3-4efb-975d-79c6e3533742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:32.010996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 309/609] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:32.038107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47477a08-8c4d-4495-b822-b4839c7d46f7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:32.154910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47477a08-8c4d-4495-b822-b4839c7d46f7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:32.197888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:32.198562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:32.198841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:32.199113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:32.211916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:32.218652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:32.226050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:32.226904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 301/610] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:32.227656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b827a0-c9a3-4efb-975d-79c6e3533742 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:32.284294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:32.389574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b827a0-c9a3-4efb-975d-79c6e3533742 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:32.443272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62889711-9684-47c2-8d4f-26205d49355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:32.443836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62889711-9684-47c2-8d4f-26205d49355e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:32.444123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62889711-9684-47c2-8d4f-26205d49355e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:32.444401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62889711-9684-47c2-8d4f-26205d49355e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:32.445177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62889711-9684-47c2-8d4f-26205d49355e None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:32.445488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62889711-9684-47c2-8d4f-26205d49355e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:32.446885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62889711-9684-47c2-8d4f-26205d49355e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:32.462325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 310/611] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:32 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:32.468319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:32.472183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8411f-1437-4858-b5d7-ec8b1af8c9bb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:32.472491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8411f-1437-4858-b5d7-ec8b1af8c9bb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:32.472800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8411f-1437-4858-b5d7-ec8b1af8c9bb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:32.480491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 311/612] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:32.480491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a66701-6a2e-4730-8fda-0dbeb8e42970 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:32.480491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a66701-6a2e-4730-8fda-0dbeb8e42970 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:32.480491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a66701-6a2e-4730-8fda-0dbeb8e42970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:32.501086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e7a66701-6a2e-4730-8fda-0dbeb8e42970 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:32.502421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e7a66701-6a2e-4730-8fda-0dbeb8e42970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:32.532636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:32.577814 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 312/613] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:16:32.604802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a66701-6a2e-4730-8fda-0dbeb8e42970 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:32.632129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:32.712595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d150ee-02ee-4932-8450-475a730384cb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:32.718312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a66701-6a2e-4730-8fda-0dbeb8e42970 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:32.755946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:32.756653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:32.756941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:32.757285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:32.770707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:32.775878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:32.776902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:32.777743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 313/614] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:32.784219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:32.803235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:32.803973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:32.804254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:32.804580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:32.816685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:32.818143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:32.834278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:32.836385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:32.838505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:32.838505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:32.838505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:32.838505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:32.838505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 302/615] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:32.850025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:32.887742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:32.902386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997323c6-3ddf-4123-ba08-c3f05739a5a6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:32.941961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:32.942707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:32.943002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:32.943283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:32.944032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:32.944344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:32.945913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:32.956322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 314/616] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:32 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:32.961970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:32.968289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.001694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106b5f4-dd58-4057-a3c1-32e84d8b7b85 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.038750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.039520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.039677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.039915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.040639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:33.040949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.042477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.055525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 303/617] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:33.061009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.074735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b7bd9-e996-426d-aabe-c6fe1ac7dfe2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.115343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.116012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.116302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.116880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.129633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.131108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.141556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:33.150244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:33.152307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:33.154669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:33.154669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:33.154669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:33.154669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:33.154669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 315/618] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:33.157724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.259353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269aa7-a719-4454-9bf9-66438b985651 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.294128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce0027-656e-40fe-87eb-ea419b3cc39f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.302700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.310910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c3a1d-f47e-4663-9772-c8050233be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.311592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c3a1d-f47e-4663-9772-c8050233be7d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.311889 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c3a1d-f47e-4663-9772-c8050233be7d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.312328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c3a1d-f47e-4663-9772-c8050233be7d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.321566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c3a1d-f47e-4663-9772-c8050233be7d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.323756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c3a1d-f47e-4663-9772-c8050233be7d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-927c3a1d-f47e-4663-9772-c8050233be7d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:33.339011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:33.340184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:33.341544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:33.342108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4afc6-2afe-4b85-8ba4-4d2b287c69aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.342199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 304/619] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:33.342275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4afc6-2afe-4b85-8ba4-4d2b287c69aa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.342275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4afc6-2afe-4b85-8ba4-4d2b287c69aa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.342275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4afc6-2afe-4b85-8ba4-4d2b287c69aa None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.342471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fc911-fb48-464e-9cdc-666de05be3b4 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.342471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fc911-fb48-464e-9cdc-666de05be3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.342471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fc911-fb48-464e-9cdc-666de05be3b4 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.342653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4afc6-2afe-4b85-8ba4-4d2b287c69aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:33.342653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4afc6-2afe-4b85-8ba4-4d2b287c69aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.344311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 305/620] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:33.345655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4afc6-2afe-4b85-8ba4-4d2b287c69aa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.347143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fc911-fb48-464e-9cdc-666de05be3b4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.360272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 316/621] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:33.361711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac572-e109-4e44-b84f-eadab81d4f03 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.362001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac572-e109-4e44-b84f-eadab81d4f03 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.362283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac572-e109-4e44-b84f-eadab81d4f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.385948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-19bac572-e109-4e44-b84f-eadab81d4f03 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:33.387566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-19bac572-e109-4e44-b84f-eadab81d4f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:33.465926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.481886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 317/622] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:33.482911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac572-e109-4e44-b84f-eadab81d4f03 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.506021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fc911-fb48-464e-9cdc-666de05be3b4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.555190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.556330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.556821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.557821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.559063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:33.559556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.562407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.592056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 306/623] 10.222.0.128 () {56 vars in 1140 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:33.593257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.602233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac572-e109-4e44-b84f-eadab81d4f03 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.644048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.644880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.645265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.645644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.653412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.654976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.662987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:33.668165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:33.669333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 15:16:33.670516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:33.677101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 318/624] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:33.677101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.733992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.747612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a3bc-fd29-4a2a-9a36-511c5c7e28d0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.794997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238597d-0c4c-462a-96e0-5690c031251e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.797255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.797912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.798197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.798474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.810922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:33.817494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:33.818890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:33.819919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:33.820775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 307/625] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:33.821300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.839663 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb21c6e-b4e9-4cbb-8c16-0de7fffadbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.840311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb21c6e-b4e9-4cbb-8c16-0de7fffadbee None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.840647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb21c6e-b4e9-4cbb-8c16-0de7fffadbee None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.840919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb21c6e-b4e9-4cbb-8c16-0de7fffadbee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.852455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb21c6e-b4e9-4cbb-8c16-0de7fffadbee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.853944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb21c6e-b4e9-4cbb-8c16-0de7fffadbee None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ceb21c6e-b4e9-4cbb-8c16-0de7fffadbee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:33.869568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:33.870527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:33.871402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:33.871402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:33.871402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:33.871402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:33.871402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 319/626] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:33.872367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebdc78-7103-44d3-b22b-43f3109c16d4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.872367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebdc78-7103-44d3-b22b-43f3109c16d4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.872538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebdc78-7103-44d3-b22b-43f3109c16d4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.877177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 320/627] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:33 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:33.878840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebdc78-7103-44d3-b22b-43f3109c16d4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:33.936119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a1b3a-5ede-4d92-8153-33dd588a8326 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:33.948878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.974163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:33.974974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:33.975425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:33.977062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:33.977877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:33.978190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:33.979757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:33.986519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:33.990583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebdc78-7103-44d3-b22b-43f3109c16d4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.001422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 308/628] 10.222.0.128 () {56 vars in 1142 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:34.006253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.034684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b48b0c-0df8-4337-be02-3362b9c894f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.035613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b48b0c-0df8-4337-be02-3362b9c894f0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.035897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b48b0c-0df8-4337-be02-3362b9c894f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.036450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b48b0c-0df8-4337-be02-3362b9c894f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.037625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b48b0c-0df8-4337-be02-3362b9c894f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:34.038199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b48b0c-0df8-4337-be02-3362b9c894f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.039573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b48b0c-0df8-4337-be02-3362b9c894f0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.056325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 321/629] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:33 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:34.056325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7307ef-52e7-49b6-b80c-c71e2ebe7db3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.056325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7307ef-52e7-49b6-b80c-c71e2ebe7db3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.056325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7307ef-52e7-49b6-b80c-c71e2ebe7db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.074954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-af7307ef-52e7-49b6-b80c-c71e2ebe7db3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:34.076049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-af7307ef-52e7-49b6-b80c-c71e2ebe7db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:34.110856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c770-3a01-4506-bcfd-c268ed4b8d13 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.147988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.148656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.148945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.156663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.156865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 322/630] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:34.156969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:34.157082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7307ef-52e7-49b6-b80c-c71e2ebe7db3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.157188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.157188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.172059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 309/631] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:34.173660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.205942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:34.278389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:34.285642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ae7bd-1d8f-4305-bd1f-e32f9edfcf5f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.290063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7307ef-52e7-49b6-b80c-c71e2ebe7db3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.333197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.333599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.333745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.334001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.334251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.334351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.334533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.334818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.347630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.349139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:34.363996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:34.365167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:34.366160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:34.366160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:34.366160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:34.366160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:34.366160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 310/632] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:34.366160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.406027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.407512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.452510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:34.472594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5637d-dbd5-446f-9aeb-e756ce316a5c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.506239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:34.511499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09a0e6-a495-4728-883e-be9ccdd26f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.512152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09a0e6-a495-4728-883e-be9ccdd26f43 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.512516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09a0e6-a495-4728-883e-be9ccdd26f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.517580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09a0e6-a495-4728-883e-be9ccdd26f43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.517706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 323/633] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:34.517706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.529565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09a0e6-a495-4728-883e-be9ccdd26f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0b09a0e6-a495-4728-883e-be9ccdd26f43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:34.535671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:34.536935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 311/634] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:34.538226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108043-e1c3-48d1-b0ab-3dd4588325b9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.538978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108043-e1c3-48d1-b0ab-3dd4588325b9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.538978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108043-e1c3-48d1-b0ab-3dd4588325b9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.540169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 312/635] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:34 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:34.540724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23108043-e1c3-48d1-b0ab-3dd4588325b9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.639744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6e0ed-455d-491e-8b15-b7490f77a705 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.651568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23108043-e1c3-48d1-b0ab-3dd4588325b9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.676489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.677163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.677448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.677726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.684355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.685834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.686085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.686703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.687039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.687394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.688138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:34.688446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.689834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:34.698428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:34.699644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:34.700584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:34.701037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:34.701110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 324/636] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:34.701110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.705878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 313/637] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:34.706711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.754400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:34.803578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fb267-aae5-45e1-a397-ab27297df621 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.816791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dfce-0dbc-4131-9bf4-93a41ae63011 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:34.840769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.841409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.841688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.841963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.851078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f19dca-c520-423c-ac3c-a8e5a75770c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:34.851692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f19dca-c520-423c-ac3c-a8e5a75770c5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.851974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f19dca-c520-423c-ac3c-a8e5a75770c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.852245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f19dca-c520-423c-ac3c-a8e5a75770c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.852984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f19dca-c520-423c-ac3c-a8e5a75770c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:34.853099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.853290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f19dca-c520-423c-ac3c-a8e5a75770c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:34.854427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.854620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f19dca-c520-423c-ac3c-a8e5a75770c5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:34.863865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 314/638] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:34.865267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83bbf8-5530-4d64-b3a2-6d38172f835d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:34.865554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83bbf8-5530-4d64-b3a2-6d38172f835d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:34.865836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83bbf8-5530-4d64-b3a2-6d38172f835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:34.876189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:34.878119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:34.879934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:34.879934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:34.879934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:34.879934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:34.879934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 325/639] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:34.879934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:34.887385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bf83bbf8-5530-4d64-b3a2-6d38172f835d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:34.888623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bf83bbf8-5530-4d64-b3a2-6d38172f835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:34.957540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:34.974077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 315/640] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:34.974922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83bbf8-5530-4d64-b3a2-6d38172f835d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.002412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111673e-3474-4532-9d47-305e6fd380f8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.004844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:35.039017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.039634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.039913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.040186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.040993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:35.041291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.042731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.061578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 326/641] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:35.062701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.105191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83bbf8-5530-4d64-b3a2-6d38172f835d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.141254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.141932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.142175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.142453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.149673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.151123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.163588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf59f1-8b7e-4478-8391-a44a7094d6c9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:35.164607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:35.166509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 316/642] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:16:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:35.168223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.207097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.208767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.208767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.208767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.220050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.221610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.226609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:35.237015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:35.237891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:35.238901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:35.238901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:35.238901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:35.238901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:35.238901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 327/643] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:35.239194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.260406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:35.276866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77326f4a-48c1-4925-9483-6bd093cf2ce7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.316237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.316917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.317256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.317464 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.318173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:35.318478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.319847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.333782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 317/644] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:35.334607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.363779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd745d-bdd2-491c-8fe1-0c79be85b2d8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.399245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.399987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.400269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.400583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.415706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:35.421245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:35.423501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 328/645] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:35.424647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.446334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2225e0c-947b-45fd-b258-a48cefc30506 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.466278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:35.490512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.491551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.491841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.492122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/97295dc4a04143fe879bef062964e4e0/roles/224e0c1f1992436bbcf1ba18913cb81b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.509447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=97295dc4a04143fe879bef062964e4e0, role_id=224e0c1f1992436bbcf1ba18913cb81b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.511185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.530283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:35.556814 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb3be1-c13f-47b6-85c3-13219e8f5018 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.577341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 318/646] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:35 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/97295dc4a04143fe879bef062964e4e0/roles/224e0c1f1992436bbcf1ba18913cb81b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:35.578428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.595513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.596185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.596508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.596791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.597769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:35.598093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.599811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.639554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 329/647] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:35.639554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.726996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa2230-1689-471e-8af8-13a65dfb8371 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.773764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427cd29-be4a-4306-9c75-1ae4f1e1b4e6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.775349 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.775988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.776273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.776576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.777287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:35.777595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.778972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.795466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 319/648] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:35.795466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.821826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.822496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.822939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.823627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.825712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:35.832088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:35.832527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.834006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:35.846411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:35.847365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:35.848351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 330/649] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:35.848957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:35.933606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d8dd3-9ea9-431a-be76-1c2c6eccc327 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.973439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf1f89-945b-46ea-ad3b-c8b2134b6130 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:35.976077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:35.976740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:35.977075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:35.977298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:35.978022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:35.978370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:35.979700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:35.993230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 320/650] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:35.993230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.022051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.023972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.024259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.024559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.090245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.094972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.103126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.104656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.120328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2a52-ea43-44f3-813c-fb23c6230b91 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.163996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.164662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.164998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.165283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.173764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.175213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:36.190234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:36.192097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 321/651] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:36.194102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.223376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 331/652] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:35 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:36.223376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.310388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee69fe0-6ff6-4dc6-ac6b-ad2c5103b460 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.336108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.344112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e372e3-27df-4ae4-808c-f873064253f7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.347901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.348548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.348831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.349108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.349867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:36.350177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.351597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.365783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 322/653] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:36.366881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.374690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.384014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.387394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.387394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.387394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.399161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.401076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:36.418682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:36.419613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:36.425971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:36.425971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:36.425971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:36.425971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:36.425971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 332/654] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:36.425971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.490453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7629a4-9879-4e2d-91ca-301814310284 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.527119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.527882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.528243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.528549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.529350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:36.529666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.531633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.535970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177922-bf9c-4bd0-b799-f935610ff469 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.549035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 323/655] 10.222.0.128 () {56 vars in 1142 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:36.549848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.575910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.576492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.576789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.577068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.590415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.592625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.602056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:36.613753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:36.619243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:36.620153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:36.620153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:36.620153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:36.620153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:36.620385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.620476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 333/656] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:36.620476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.657878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d692a4f-7002-4547-af26-8162b2d7f9da None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.697458 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.698084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.698369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.698645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.699441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=0207287721f5489f8ac7a258356e57bd) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:36.699758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.701222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.716516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 324/657] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/role_assignments?user.id=0207287721f5489f8ac7a258356e57bd&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 610 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:36.716516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.740848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c7e9c-2ece-4f10-9c72-c9749405e429 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.783554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.784291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.784689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.785039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.785823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:36.786264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.787777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:36.801737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 334/658] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:36.802752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:36.871300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c52698-444d-4b26-9e6e-74b8970fedcc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.880931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.904309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.921605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.922647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.922935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.923217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.941319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda2d1-df35-4f14-9695-46317ce72409 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:36.989309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:36.990194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:36.990480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:36.990760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:36.993180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:36.993180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:36.993500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.007010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 335/659] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:37.007855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.024248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.026553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.111659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d38bd5-d514-4097-ac5a-6c47576185da None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.132920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:37.149518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.150879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.151164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.151445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.161880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.163639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.168363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 325/660] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:36 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:37.170219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:37.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:37.181594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 336/661] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:37.182531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.201981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:37.286847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6ed58-ea0f-4321-a311-031fccdc7bb2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.299702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165831a7-3048-46e2-9ae0-e3a355777ef2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.327358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.328683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.328683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.329389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.337708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.339207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.339415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.340190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.340646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.341078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:37.352713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:37.353897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:37.354962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:37.355936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 326/662] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:37.356824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.357711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.361467 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:37.378338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:37.379422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:37.380324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:37.380324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:37.380324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:37.380324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:37.382140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 337/663] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:37.383143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.418849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:37.456831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:37.474454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4bb0e-e779-4824-a521-3e662e8c68b9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.501507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c7ac4-f3d6-4090-b78d-ed6798ad8b25 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.512936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.513577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.513983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.514208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.515903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:37.516570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.517837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.542540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 327/664] 10.222.0.128 () {56 vars in 1144 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:37.543594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.544518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.545433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.545862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.546401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.547987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:37.548482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.550135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.565005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 338/665] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:37.566145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.654485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9860cdb-f8c6-473d-9b79-37545b3aeead None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.668986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:37.680208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b8c62-92a3-4c18-9b11-2019c9e23476 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.694186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.694742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.694997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.695274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.695974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:37.696283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.697674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.711348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 328/666] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:37.712181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.717756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:37.723488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a07803-e56e-49a2-99bb-dc9b12e1309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.724119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a07803-e56e-49a2-99bb-dc9b12e1309a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.724402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a07803-e56e-49a2-99bb-dc9b12e1309a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.724712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a07803-e56e-49a2-99bb-dc9b12e1309a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.805887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a07803-e56e-49a2-99bb-dc9b12e1309a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.807353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a07803-e56e-49a2-99bb-dc9b12e1309a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.830822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91695-cd63-4108-b268-5a0a4dd69772 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:37.876719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e33cf4-4960-4643-a282-a75c54ad8238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:37.877752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e33cf4-4960-4643-a282-a75c54ad8238 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.878223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e33cf4-4960-4643-a282-a75c54ad8238 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.878517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e33cf4-4960-4643-a282-a75c54ad8238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.879670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e33cf4-4960-4643-a282-a75c54ad8238 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=92e40a33db1a41d59d9a7a3bef5f3ece) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:37.879984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e33cf4-4960-4643-a282-a75c54ad8238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:37.881472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e33cf4-4960-4643-a282-a75c54ad8238 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:37.896549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 329/667] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/role_assignments?user.id=92e40a33db1a41d59d9a7a3bef5f3ece&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 610 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:37.896549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3ab6-3180-4fd1-a9b0-754bf21edc1b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.896549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3ab6-3180-4fd1-a9b0-754bf21edc1b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.896549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3ab6-3180-4fd1-a9b0-754bf21edc1b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.896549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 330/668] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:37 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:37.896549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3ab6-3180-4fd1-a9b0-754bf21edc1b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:37.941727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:37.954953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 339/669] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:37.956744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df09c47-eac9-409d-80dc-ffbca061ed72 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:37.957221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df09c47-eac9-409d-80dc-ffbca061ed72 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:37.957672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df09c47-eac9-409d-80dc-ffbca061ed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:37.979851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4df09c47-eac9-409d-80dc-ffbca061ed72 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:37.981419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4df09c47-eac9-409d-80dc-ffbca061ed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:38.019549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:38.071734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3ab6-3180-4fd1-a9b0-754bf21edc1b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.076431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 340/670] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:38.078122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df09c47-eac9-409d-80dc-ffbca061ed72 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.125307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.126443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.126754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.127029 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.134300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.136636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:38.150031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:38.150947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:38.151817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 331/671] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:38.152299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.217472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df09c47-eac9-409d-80dc-ffbca061ed72 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.241693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:38.261573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.262451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.262564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.262778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.277729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.279112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.279216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bedeb0-f4d3-49b6-a431-568f19e405d9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:38.303670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:38.304672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:38.305547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:38.305547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:38.305547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:38.305547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:38.305547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 341/672] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:38.306372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.311948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:38.318548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.319276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.322141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.322141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.331468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:38.336829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:38.338860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 332/673] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:38.341010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.429171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027b819-9135-4d36-a94e-9e2e73bd0ad2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.451802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9229-bb46-44ff-a21f-a846e925f40f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.468026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.472488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.472488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.472488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.472488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:38.472488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.473686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.491803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 342/674] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:38.492848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.497830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.498467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.498756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.499038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.500287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:38.500622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.502007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.520561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 333/675] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:38.521572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.536057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:38.591027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:38.617690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3ac46-e718-4fa8-a6b5-0a455db5655c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.646347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cb307-b401-463b-a861-5b0b7fec2954 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.661619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.662541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.662970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.663415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.664402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:38.664896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.666464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.681033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 343/676] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:38.682319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.683949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.684613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.684905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.685184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.685990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=8ff6f527aa364534b2fdd0b770924e83) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:38.686308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.687962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.694628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 334/677] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/role_assignments?user.id=8ff6f527aa364534b2fdd0b770924e83&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 610 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:38.695775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.793649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:38.802812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485f7f9-3afe-482a-bf3c-62766128fe09 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.808437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f5c87-baea-451c-80ef-b1ff69f7a2d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:38.822687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:38.844407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.845670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.845994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:38.846425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.846653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:38.846943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:38.847222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.847504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:38.856863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.859427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.860712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:38.862269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:38.873865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:38.875397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:38.876311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:38.878035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 344/678] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:38.884887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:38.885830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:38.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:38.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:38.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:38.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:38.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 335/679] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:38.889607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:38.893546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.008540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21671e29-0247-40b7-b4ff-e77d2b7ea16b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:39.049727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297696d4-f2fe-4351-b50f-0e45ce95065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:39.050371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297696d4-f2fe-4351-b50f-0e45ce95065a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.050735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297696d4-f2fe-4351-b50f-0e45ce95065a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.051082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297696d4-f2fe-4351-b50f-0e45ce95065a None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.051915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297696d4-f2fe-4351-b50f-0e45ce95065a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:39.052294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297696d4-f2fe-4351-b50f-0e45ce95065a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:39.053311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18de1-abb7-4bc3-bca8-27adb5146b7c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:39.054023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297696d4-f2fe-4351-b50f-0e45ce95065a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:39.072549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 345/680] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:39.073843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e31841-7c64-404f-9e4b-72462cdcb743 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.074134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e31841-7c64-404f-9e4b-72462cdcb743 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.074416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e31841-7c64-404f-9e4b-72462cdcb743 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.075902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 346/681] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:39.077566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e31841-7c64-404f-9e4b-72462cdcb743 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.095910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:39.098487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd58c5f-0da5-47ab-9ce3-f26cbbf491bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:39.099316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58c5f-0da5-47ab-9ce3-f26cbbf491bc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.099763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58c5f-0da5-47ab-9ce3-f26cbbf491bc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.100182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58c5f-0da5-47ab-9ce3-f26cbbf491bc None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.101064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd58c5f-0da5-47ab-9ce3-f26cbbf491bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:39.101545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd58c5f-0da5-47ab-9ce3-f26cbbf491bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:39.103042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd58c5f-0da5-47ab-9ce3-f26cbbf491bc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:39.117321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 336/682] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:39.118943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814eed9a-98a8-430a-af67-e1dbc6770e9f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.119378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814eed9a-98a8-430a-af67-e1dbc6770e9f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.119903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814eed9a-98a8-430a-af67-e1dbc6770e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.141792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-814eed9a-98a8-430a-af67-e1dbc6770e9f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:39.143115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-814eed9a-98a8-430a-af67-e1dbc6770e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:39.165867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:39.223941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 337/683] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:39.229067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814eed9a-98a8-430a-af67-e1dbc6770e9f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.239576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e31841-7c64-404f-9e4b-72462cdcb743 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:39.288925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:39.289550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.289833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.290105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.290963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=c6ecc741022f4d6189f4d3f19b4cdbe0) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:39.291277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:39.292899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:39.303066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 347/684] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:39 2022] GET /identity/v3/role_assignments?user.id=c6ecc741022f4d6189f4d3f19b4cdbe0&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 610 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:39.304140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.403363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814eed9a-98a8-430a-af67-e1dbc6770e9f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:39.459400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:39.460320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.460777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.461238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.475931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:39.477416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:39.496827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:39.500917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:39.502052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:39.503425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:39.506995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 338/685] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:39.507868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.532881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63237d-0afb-4fb3-8139-47f9b3bcc2bf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:39.533930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:39.586119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:39.586686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.586965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.587239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.602297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:39.604038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:39.637015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:39.639131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:39.641946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:39.641946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:39.641946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:39.641946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:39.648558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 348/686] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:39.661010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.676979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fc6fb-f77d-403b-9d57-25b8c566bc99 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:39.744686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:39.745314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.745597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.745874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.746680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:39.747019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:39.748487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:39.791792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 339/687] 10.222.0.128 () {56 vars in 1140 bytes} [Tue Jul 26 15:16:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:39.797729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.831892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da588f8-96a9-4a0f-92e6-1b222e31304c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:39.900923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:39.910647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:39.911450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:39.911787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:39.912383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:39.913154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:39.913467 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:39.914855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:39.953450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:39.956868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 349/688] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:39.965470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:39.966241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d64a51-b4d9-4deb-9093-73b022cded10 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:40.002651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:40.003454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.003840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.004226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.095251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:40.096961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:40.130093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9508a94-ee37-4734-9fc7-b8be69199cf8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:40.176945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:40.198238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:40.198892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.199176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.199452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.200164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:40.201684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:40.218435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 340/689] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:40.224533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:40.297308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 350/690] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:16:39 2022] POST /identity/v3/users => generated 255 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:16:40.343305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167010c-6396-4ea3-b434-75fae05e9164 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:40.381580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539bf06-44e2-44b6-b44a-007a9c56b32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:40.382323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bf06-44e2-44b6-b44a-007a9c56b32b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.382606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bf06-44e2-44b6-b44a-007a9c56b32b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.382884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bf06-44e2-44b6-b44a-007a9c56b32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.395407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539bf06-44e2-44b6-b44a-007a9c56b32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:40.396909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539bf06-44e2-44b6-b44a-007a9c56b32b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9539bf06-44e2-44b6-b44a-007a9c56b32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:40.413765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:40.415125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:40.416602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:40.416602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:40.416602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:40.416602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:40.416602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 341/691] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:40.420730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:40.508797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd013-63c9-4c1a-9158-e92b1583988a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.509080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd013-63c9-4c1a-9158-e92b1583988a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.509358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd013-63c9-4c1a-9158-e92b1583988a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.514339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 342/692] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:40.516012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb6f7-3a57-4f94-a715-d3f37b447b8b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.516560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb6f7-3a57-4f94-a715-d3f37b447b8b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.517072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb6f7-3a57-4f94-a715-d3f37b447b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.526040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:40.542644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4d9eb6f7-3a57-4f94-a715-d3f37b447b8b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:40.543932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4d9eb6f7-3a57-4f94-a715-d3f37b447b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:40.563771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:40.588197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70354a-a317-4d4d-8a96-6a59dba80bea None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:40.623406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 343/693] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:40.625814 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:40.626670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.627125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.627623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.628549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:40.629031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:40.630728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:40.645027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 351/694] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:40.647761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eb6f7-3a57-4f94-a715-d3f37b447b8b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:40.651734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:40.761180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687baa5-eb62-40b7-bffa-d58d957848a1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:40.764663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eb6f7-3a57-4f94-a715-d3f37b447b8b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:40.774329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:40.800621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:40.802118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.802667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.803133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.804479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:40.805210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:40.805568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:40.805915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:40.811905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:40.814133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:40.817872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:40.822915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:40.823857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:40.824716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 344/695] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:40.830417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:40.830729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:40.831598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:40.832521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:40.835160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 352/696] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:40.841552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:40.858026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:40.968967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce6d00-e0f4-4d76-8e1f-c1c90a8c52f9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.005334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:41.006364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.006896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.007394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.008392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:41.008976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:41.011114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:41.024894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 353/697] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:41.033060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:41.051809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdfef3-1e9d-48ce-b519-3f04cde8556d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.088808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:41.121754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:41.122401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.122688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.122967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.123721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:41.124007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:41.126441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:41.148016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 345/698] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:40 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:41.153414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:41.175816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d1dec-f44a-4769-bbf3-a901d9c36d3c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.213888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a478d-4765-4b9d-a685-f4a53f24d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:41.214915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a478d-4765-4b9d-a685-f4a53f24d3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.215435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a478d-4765-4b9d-a685-f4a53f24d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.215909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a478d-4765-4b9d-a685-f4a53f24d3c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.216992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a478d-4765-4b9d-a685-f4a53f24d3c2 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=97295dc4a04143fe879bef062964e4e0) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:41.218591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a478d-4765-4b9d-a685-f4a53f24d3c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:41.219368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a478d-4765-4b9d-a685-f4a53f24d3c2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:41.226513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 354/699] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:41 2022] GET /identity/v3/role_assignments?user.id=97295dc4a04143fe879bef062964e4e0&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 610 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:41.271386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:41.299134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83569357-ca2f-4ee3-8255-b5ebb3be1c2b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.321599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001394f8-0f2e-44ce-a999-0df367615ba2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.321887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001394f8-0f2e-44ce-a999-0df367615ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.322167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001394f8-0f2e-44ce-a999-0df367615ba2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.323645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 355/700] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:41.331440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5074-1da6-4674-a1cd-4fb11f6dab94 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.331894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5074-1da6-4674-a1cd-4fb11f6dab94 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.332247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5074-1da6-4674-a1cd-4fb11f6dab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.353652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:41.354613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.355236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.355796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.356516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cb6d5074-1da6-4674-a1cd-4fb11f6dab94 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:41.357785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cb6d5074-1da6-4674-a1cd-4fb11f6dab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:41.384812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:41.387866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:41.418350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:41.419758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:41.420663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:41.420663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:41.420663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:41.420663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:41.420663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 346/701] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:41.429624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:41.516243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 356/702] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 15:16:41.552319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5074-1da6-4674-a1cd-4fb11f6dab94 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:41.577529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c3b8f-8c06-4d50-b914-7e8890772f46 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.593779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:41.618761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:41.629665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:41.631472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.632071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.632636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.635243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:41.635243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:41.635243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:41.649826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 347/703] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:41.656590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:41.718853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5074-1da6-4674-a1cd-4fb11f6dab94 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.777571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:41.778486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.780002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.780554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.781387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:41.782965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:41.794618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:41.795884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:41.797361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:41.800131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 357/704] 10.222.0.128 () {56 vars in 1134 bytes} [Tue Jul 26 15:16:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:41.806156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:41.807283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1423c-f053-4b07-9c9c-1b6d4f2bffcb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.876310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:41.877424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:41.877917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:41.878400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:41.892634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:41.894587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:41.916496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:41.917527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:41.918444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 348/705] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:16:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:41.923307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:41.972592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:41.988892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b05d-ab72-4b23-a46b-c2c38ad3ab6c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:41.997218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:42.029333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d186d-cd5f-4263-b6d2-8d333798f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.030070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d186d-cd5f-4263-b6d2-8d333798f4ab None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.030411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d186d-cd5f-4263-b6d2-8d333798f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.031199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d186d-cd5f-4263-b6d2-8d333798f4ab None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.031894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d186d-cd5f-4263-b6d2-8d333798f4ab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:42.032205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d186d-cd5f-4263-b6d2-8d333798f4ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.033668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d186d-cd5f-4263-b6d2-8d333798f4ab None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:42.044555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 358/706] 10.222.0.128 () {56 vars in 1129 bytes} [Tue Jul 26 15:16:41 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:42.044555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa30c8-415d-4797-b11c-9dabfcf11e21 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.044555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa30c8-415d-4797-b11c-9dabfcf11e21 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.044555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa30c8-415d-4797-b11c-9dabfcf11e21 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.048276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 359/707] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:42.049170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa30c8-415d-4797-b11c-9dabfcf11e21 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:42.075349 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8cd12-11a6-4a76-a1dd-474eb2a9371e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:42.130289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309382b-b2e7-4169-9b4a-e474dcbe4dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.131982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309382b-b2e7-4169-9b4a-e474dcbe4dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.132779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309382b-b2e7-4169-9b4a-e474dcbe4dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.133281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309382b-b2e7-4169-9b4a-e474dcbe4dc3 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.135385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309382b-b2e7-4169-9b4a-e474dcbe4dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:42.135908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309382b-b2e7-4169-9b4a-e474dcbe4dc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.137556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309382b-b2e7-4169-9b4a-e474dcbe4dc3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:42.163576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 349/708] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:16:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:42.165029 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdbd42-a1bf-45f2-94f0-217cdd9601eb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.165328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdbd42-a1bf-45f2-94f0-217cdd9601eb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.165617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdbd42-a1bf-45f2-94f0-217cdd9601eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.175896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa30c8-415d-4797-b11c-9dabfcf11e21 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:42.184310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8ecdbd42-a1bf-45f2-94f0-217cdd9601eb None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:42.185557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ecdbd42-a1bf-45f2-94f0-217cdd9601eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:42.225541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.227354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.227820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.228296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.229194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:42.229679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.231301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:42.242055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 360/709] 10.222.0.128 () {56 vars in 1129 bytes} [Tue Jul 26 15:16:42 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:42.243124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:42.261794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:42.290241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 350/710] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:42.324311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdbd42-a1bf-45f2-94f0-217cdd9601eb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:42.373179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31347e3-6e88-4879-8cfe-40ab95793684 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:42.422193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.423468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.423930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.424369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.426862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:42.525561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.527469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:42.544260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdbd42-a1bf-45f2-94f0-217cdd9601eb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:42.544689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:42.583453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.584112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.584405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.584699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.585342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.586714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:42.598695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:42.599890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:42.600762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 351/711] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:42 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:42.605673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:42.654422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 361/712] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:42 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:42.661646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:42.717961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaaec9-c20a-49e8-acbd-ef8cc3512da6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:42.718673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:42.768347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.769016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.769300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.769581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.770281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] RBAC: Adding query filter params (name=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:42.770594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.771992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:42.781558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 352/713] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:16:42 2022] GET /identity/v3/services?name=network => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:42.782800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:42.858595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23137-7995-40a1-a180-61fca5b58914 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:42.886757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:42.890909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51132b-f7c4-4bd5-9036-b95ec32f8273 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:42.927178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.927837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.928191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.928591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.929707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] RBAC: Adding query filter params (type=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:42.930091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.931660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:42.944613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 353/714] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:16:42 2022] GET /identity/v3/services?type=network => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:42.945919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d497-db13-4f19-a905-095b1e057919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:42.946613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d497-db13-4f19-a905-095b1e057919 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.946903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d497-db13-4f19-a905-095b1e057919 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.947197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d497-db13-4f19-a905-095b1e057919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.979009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6d497-db13-4f19-a905-095b1e057919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:42.980557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6d497-db13-4f19-a905-095b1e057919 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0df6d497-db13-4f19-a905-095b1e057919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:43.009744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:43.010696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:43.011600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:43.011600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:43.011600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:43.011600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:43.011600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 362/715] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:43.017021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.076232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc0144-529a-4a18-82a4-f5801953c436 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.076553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc0144-529a-4a18-82a4-f5801953c436 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.076838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc0144-529a-4a18-82a4-f5801953c436 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.077867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:43.078762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 363/716] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:43.084267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58628b-4845-4bfc-9a7a-5789dc467887 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.084583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58628b-4845-4bfc-9a7a-5789dc467887 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.084869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58628b-4845-4bfc-9a7a-5789dc467887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.117670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fa58628b-4845-4bfc-9a7a-5789dc467887 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:43.118919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fa58628b-4845-4bfc-9a7a-5789dc467887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:43.192770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908fdfd-bb0a-47d6-8808-221bd8328c6e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.215408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 364/717] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:43.238294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.239174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.240407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.241204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.242248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:43.242670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.243742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58628b-4845-4bfc-9a7a-5789dc467887 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.244564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.269495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 354/718] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:43.273292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.428829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:43.438612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58628b-4845-4bfc-9a7a-5789dc467887 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.456949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:43.482147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bdbdc-5fa8-4ee0-854c-886fb7a85b5c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.513641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.514390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.514934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.515747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.536129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:43.542414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:43.543370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:43.544220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:43.549473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 365/719] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:43.550550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.551201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.551480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.551750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.555666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.569346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.570973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:43.600141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:43.601167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:43.602134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 355/720] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:43.607386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.682364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da476be1-550a-4506-bb03-f59268b82b5a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.738012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.738970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.739409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.739828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.740975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:43.742414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.744125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.763242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 366/721] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:43.772804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.807346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:43.813168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3616a1f-630f-4698-8c23-a9ddecec3f50 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.869280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.869984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.878775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.878775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.878775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:43.878775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.878775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.891556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 356/722] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:43.901494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.923485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:43.933496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa11df5-4869-4bf9-b029-a933871ed46a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.971705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.972849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.973292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.973722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.989533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.992609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:44.012738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:44.013847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:44.015463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:44.015463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:44.015463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:44.015463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:44.018075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 367/723] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:44.023719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:44.037935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1306068-6e93-4b87-a673-86cbb25b8a44 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:44.074248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85a76d-28d1-4bf0-be56-d4f33010e386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:44.075021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85a76d-28d1-4bf0-be56-d4f33010e386 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.075411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85a76d-28d1-4bf0-be56-d4f33010e386 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.075902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85a76d-28d1-4bf0-be56-d4f33010e386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.077633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85a76d-28d1-4bf0-be56-d4f33010e386 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=92e40a33db1a41d59d9a7a3bef5f3ece) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:44.078012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85a76d-28d1-4bf0-be56-d4f33010e386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:44.079513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85a76d-28d1-4bf0-be56-d4f33010e386 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:44.087568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 357/724] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/role_assignments?user.id=92e40a33db1a41d59d9a7a3bef5f3ece&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:44.140764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a1f8a-98dc-45ea-a23b-b937499c064f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:44.144898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:44.154472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f16e12-0f31-49f1-a38b-a805054b282e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.154890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f16e12-0f31-49f1-a38b-a805054b282e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.155881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f16e12-0f31-49f1-a38b-a805054b282e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.158208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 358/725] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:44 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:44.163441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa992c9c-8ada-4b9c-a41e-02dcecded18e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.164035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa992c9c-8ada-4b9c-a41e-02dcecded18e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.164555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa992c9c-8ada-4b9c-a41e-02dcecded18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.207585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fa992c9c-8ada-4b9c-a41e-02dcecded18e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:44.209988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa992c9c-8ada-4b9c-a41e-02dcecded18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:44.214695 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:44.217147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.219182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.219696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.220693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:44.223466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:44.225171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:44.255421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 368/726] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:44.261888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:44.353935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 359/727] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:44.384949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa992c9c-8ada-4b9c-a41e-02dcecded18e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:44.388872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:44.434203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befebe7-f367-43c8-9756-53ca7ce4b913 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:44.468508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:44.485448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:44.486123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.486464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.486745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.501654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:44.503206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:44.526288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa992c9c-8ada-4b9c-a41e-02dcecded18e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:44.529992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:44.531508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:44.533036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 369/728] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:16:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:44.539333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:44.565397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:44.566149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.566433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.566708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.578778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:44.584052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:44.586210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:44.588235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 360/729] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:44.592571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:44.725720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:44.756909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015faeb-4328-4b07-859d-6e2956928264 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:44.805783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d25c8-d760-481f-9f18-fa97c0182ed0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:44.820169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22f3fd-7015-4aea-b822-6208c5b39950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:44.821076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f3fd-7015-4aea-b822-6208c5b39950 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.821508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f3fd-7015-4aea-b822-6208c5b39950 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.821963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f3fd-7015-4aea-b822-6208c5b39950 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.822913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22f3fd-7015-4aea-b822-6208c5b39950 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:44.823373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22f3fd-7015-4aea-b822-6208c5b39950 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:44.824994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22f3fd-7015-4aea-b822-6208c5b39950 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:44.848632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 370/730] 10.222.0.128 () {56 vars in 1154 bytes} [Tue Jul 26 15:16:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:44.849640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98883524-318e-4865-a26f-eb567d5c3c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:44.850369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98883524-318e-4865-a26f-eb567d5c3c5d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.850542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467419-a1a4-47b6-837d-7e935af7da94 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.850832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98883524-318e-4865-a26f-eb567d5c3c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.851122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467419-a1a4-47b6-837d-7e935af7da94 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.851269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98883524-318e-4865-a26f-eb567d5c3c5d None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.851679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467419-a1a4-47b6-837d-7e935af7da94 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.852202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98883524-318e-4865-a26f-eb567d5c3c5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:44.852669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98883524-318e-4865-a26f-eb567d5c3c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:44.854249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98883524-318e-4865-a26f-eb567d5c3c5d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:44.856546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 371/731] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:44 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:44.857790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15467419-a1a4-47b6-837d-7e935af7da94 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:44.866667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 361/732] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:44 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:44.868229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0acacb-f8e8-42a0-8561-02830c55f6d9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:44.868710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0acacb-f8e8-42a0-8561-02830c55f6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:44.869120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0acacb-f8e8-42a0-8561-02830c55f6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:44.889431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8e0acacb-f8e8-42a0-8561-02830c55f6d9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:44.891493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e0acacb-f8e8-42a0-8561-02830c55f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:44.931101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:44.970915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 362/733] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:44.972191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0acacb-f8e8-42a0-8561-02830c55f6d9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:45.013774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15467419-a1a4-47b6-837d-7e935af7da94 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.052009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.053084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.053545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.054025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.055218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:45.116649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0acacb-f8e8-42a0-8561-02830c55f6d9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.130363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.132087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:45.164987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.166144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.166504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.166975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.180613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.182926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:45.183511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:45.205092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:45.206792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:45.209528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:45.209528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:45.209528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:45.209528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:45.211478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 363/734] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:45.212509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:45.275079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 372/735] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 15:16:45.276022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:45.337277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015d6db-a46e-4998-b127-0d066c6d3e7f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.345890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:45.376524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d35504-516b-4ed7-a87a-94563bf1e153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.377229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d35504-516b-4ed7-a87a-94563bf1e153 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.377770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d35504-516b-4ed7-a87a-94563bf1e153 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.378509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d35504-516b-4ed7-a87a-94563bf1e153 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.379236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d35504-516b-4ed7-a87a-94563bf1e153 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.380695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d35504-516b-4ed7-a87a-94563bf1e153 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:45.395391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8198b4-cd90-4931-b98c-7af1361c81d8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.434388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.435036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.435320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.435596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.436337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:45.437391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.438492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:45.454614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 373/736] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:45.454614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:45.479821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 364/737] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:16:45 2022] POST /identity/v3/services => generated 252 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:45.481413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f00d9-0321-47f2-8ebb-b57f3b07ba14 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.481820 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f00d9-0321-47f2-8ebb-b57f3b07ba14 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.482209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f00d9-0321-47f2-8ebb-b57f3b07ba14 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.488010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 365/738] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:45.488010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f00d9-0321-47f2-8ebb-b57f3b07ba14 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:45.594888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:45.636264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f00d9-0321-47f2-8ebb-b57f3b07ba14 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.677569 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95c78d-ada5-4ac4-8eb8-c38aeeea8df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.678245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95c78d-ada5-4ac4-8eb8-c38aeeea8df5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.678539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95c78d-ada5-4ac4-8eb8-c38aeeea8df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.678824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95c78d-ada5-4ac4-8eb8-c38aeeea8df5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.691089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95c78d-ada5-4ac4-8eb8-c38aeeea8df5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.693402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95c78d-ada5-4ac4-8eb8-c38aeeea8df5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1a95c78d-ada5-4ac4-8eb8-c38aeeea8df5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:45.711284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:45.712297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 366/739] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03965c47-fcb1-4706-af2c-ce6bf3e891d0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.714100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03965c47-fcb1-4706-af2c-ce6bf3e891d0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.714858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03965c47-fcb1-4706-af2c-ce6bf3e891d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.726677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625e435-3d23-4da0-9d16-f9979e718102 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.737504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-03965c47-fcb1-4706-af2c-ce6bf3e891d0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:45.739039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-03965c47-fcb1-4706-af2c-ce6bf3e891d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:45.760064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:45.768243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.769019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.769310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.769593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.785510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.788272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:45.808061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:45.809156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:45.810038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:45.810038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:45.810038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:45.810038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:45.812611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 374/740] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:45.813598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:45.826788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 367/741] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:45 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 15:16:45.828834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03965c47-fcb1-4706-af2c-ce6bf3e891d0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:45.928959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:45.948732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27557e87-d778-4b4f-ac74-cf3a121b29f8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.952253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03965c47-fcb1-4706-af2c-ce6bf3e891d0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:45.988644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.989609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.990026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.991124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.992073 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:45.992553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.993311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:45.994041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:45.994430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:45.994631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:45.994915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:45.995728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:45.996145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:45.997659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:46.010705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 368/742] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:46.011712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.023401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 375/743] 10.222.0.128 () {56 vars in 1140 bytes} [Tue Jul 26 15:16:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:46.025002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.036427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:46.148225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c83fe3-6ad2-4082-bc2c-206fe33e62ac None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:46.157002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b4faa-7566-4a49-9648-3d326909d3f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:46.190622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:46.193929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:46.194916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.195298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.195682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.196199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:46.196686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:46.197214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.197644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.198065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.198608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:46.207277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:46.209058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:46.222263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:46.223323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:46.224239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:46.224936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 376/744] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:46.225870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.317710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 369/745] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:16:46 2022] POST /identity/v3/services => generated 241 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:46.318661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.339291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888297f-636d-4235-88d2-af5c34fa9da3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:46.390851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ca3bf-c41f-4a1d-9f31-dc1d4cc07a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:46.391311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ca3bf-c41f-4a1d-9f31-dc1d4cc07a26 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.391601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ca3bf-c41f-4a1d-9f31-dc1d4cc07a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.391953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ca3bf-c41f-4a1d-9f31-dc1d4cc07a26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.445042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:46.447903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:46.495250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ca3bf-c41f-4a1d-9f31-dc1d4cc07a26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:46.497188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ca3bf-c41f-4a1d-9f31-dc1d4cc07a26 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:46.616641 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f1c3-14b7-4e39-82ba-c88d5bc89744 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:46.632852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 377/746] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:46 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:16:46.634371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5444a50-9bc1-427a-98c7-665f80671549 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.634787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5444a50-9bc1-427a-98c7-665f80671549 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.635190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5444a50-9bc1-427a-98c7-665f80671549 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.636858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 378/747] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:46.638936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5444a50-9bc1-427a-98c7-665f80671549 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.675387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99461c0-dbb4-46c2-a48f-75d433e522d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:46.677596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99461c0-dbb4-46c2-a48f-75d433e522d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.678549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99461c0-dbb4-46c2-a48f-75d433e522d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.679177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99461c0-dbb4-46c2-a48f-75d433e522d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.681244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99461c0-dbb4-46c2-a48f-75d433e522d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:46.682146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99461c0-dbb4-46c2-a48f-75d433e522d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:46.685407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99461c0-dbb4-46c2-a48f-75d433e522d5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:46.703919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 370/748] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:46.705839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b8b-aa72-4e0d-bca0-5012afdd84dd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.706326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b8b-aa72-4e0d-bca0-5012afdd84dd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.706752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b8b-aa72-4e0d-bca0-5012afdd84dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.730271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8b8d1b8b-aa72-4e0d-bca0-5012afdd84dd None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:46.732023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b8d1b8b-aa72-4e0d-bca0-5012afdd84dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:46.764906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:46.798887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5444a50-9bc1-427a-98c7-665f80671549 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:46.825501 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 371/749] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:46 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:46.827242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d1b8b-aa72-4e0d-bca0-5012afdd84dd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.862249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:46.863982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.864454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.865354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.885488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:46.887501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:46.910481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:46.911653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:46.912660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:46.912660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:46.912660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:46.912660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:46.916861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 379/750] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:46.924075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.938934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:46.987537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d1b8b-aa72-4e0d-bca0-5012afdd84dd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:47.034812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:47.035802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:47.036113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:47.036624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:47.038173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=f35a6f8e44fa42dca9ab2ea540ba57ac) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:47.038751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:47.041215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:47.049484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 372/751] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:46 2022] GET /identity/v3/role_assignments?user.id=f35a6f8e44fa42dca9ab2ea540ba57ac&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:47.051136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:47.105485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10013b5-bf4c-4500-886a-f15b8f17d467 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:47.175743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:47.203890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501248a4-4d31-477a-9b2b-62d8cbe7dff0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:47.225273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:47.226325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:47.226852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:47.227363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:47.246860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:47.255797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:47.256455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:47.256872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:47.257171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:47.257452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:47.257915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:47.258290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:47.260408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:47.273753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:47.274932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:47.276005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:47.278693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 380/753] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:47.278855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 373/753] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:47.285569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:47.285973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:47.422046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589d9f-a06f-4b1b-af21-3cd8f6da7b8d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:47.468621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:47.469266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:47.469560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:47.470368 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:47.471142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:47.471552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:47.472989 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:47.484033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 374/754] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:47 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:47.492297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:47.566023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117990e-2727-4e39-a6e8-2ddc18022edf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:47.613155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:47.613994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:47.614289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:47.614572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:47.641326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:47.643248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:47.662921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:47.678455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:47.685907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:47.687068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 381/755] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:47.689222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:47.738853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a905da-f0c6-40c2-8983-4ee4cc3e377f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:47.791244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:47.791907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:47.792185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:47.792477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:47.822760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:47.832489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:47.832651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:47.846878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:47.847855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:47.848900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:47.848900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:47.848900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:47.848900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:47.848900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 375/756] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:47.854687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:47.961982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d76c3-9a37-480f-a6d1-79380efe8e64 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.004418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea842964-f46f-402b-bfc0-1ffda7c6c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.005442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea842964-f46f-402b-bfc0-1ffda7c6c752 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.006094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea842964-f46f-402b-bfc0-1ffda7c6c752 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.006518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea842964-f46f-402b-bfc0-1ffda7c6c752 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.007806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea842964-f46f-402b-bfc0-1ffda7c6c752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:48.008276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea842964-f46f-402b-bfc0-1ffda7c6c752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.009936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea842964-f46f-402b-bfc0-1ffda7c6c752 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.023654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:48.028499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19710880-ea06-4bcf-9c1a-21c3f6a4b231 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.044269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 382/757] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:48.046578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002950a-b98a-4cb8-a977-b21a3e081b7c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.047935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002950a-b98a-4cb8-a977-b21a3e081b7c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.048362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002950a-b98a-4cb8-a977-b21a3e081b7c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.050249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 383/758] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:48 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:48.051176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002950a-b98a-4cb8-a977-b21a3e081b7c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:48.094924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b23b4-9e98-4ee1-bbb9-9c543c4e54e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.096274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b23b4-9e98-4ee1-bbb9-9c543c4e54e8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.096961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b23b4-9e98-4ee1-bbb9-9c543c4e54e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.097588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b23b4-9e98-4ee1-bbb9-9c543c4e54e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.098924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b23b4-9e98-4ee1-bbb9-9c543c4e54e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:48.099591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b23b4-9e98-4ee1-bbb9-9c543c4e54e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.102116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b23b4-9e98-4ee1-bbb9-9c543c4e54e8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.128938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 376/759] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:48.131535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcca7ee-44bb-4bee-81ac-fbd34b5981b7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.132181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcca7ee-44bb-4bee-81ac-fbd34b5981b7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.132832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcca7ee-44bb-4bee-81ac-fbd34b5981b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.165121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7bcca7ee-44bb-4bee-81ac-fbd34b5981b7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:48.166677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7bcca7ee-44bb-4bee-81ac-fbd34b5981b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:48.191600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002950a-b98a-4cb8-a977-b21a3e081b7c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.194564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:48.242410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.243729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.244435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.245723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.246675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=97295dc4a04143fe879bef062964e4e0) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:48.247192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.249471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.259973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 384/760] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:48 2022] GET /identity/v3/role_assignments?user.id=97295dc4a04143fe879bef062964e4e0&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:48.259973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:48.260699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 377/761] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:48 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:48.310098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcca7ee-44bb-4bee-81ac-fbd34b5981b7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:48.396692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80270a09-33ff-464f-ae30-26c233c63ed3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.476300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.477410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.479499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.479879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.501404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.503058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:48.522786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:48.523793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:48.524781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 385/762] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:16:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:48.532914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:48.534440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:48.553319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcca7ee-44bb-4bee-81ac-fbd34b5981b7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.578611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:48.608756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.609438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.609740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.610011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.611403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.612808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:48.638057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:48.639060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:48.640449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 378/763] 10.222.0.128 () {56 vars in 1134 bytes} [Tue Jul 26 15:16:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:48.648443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:48.664149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec26ca-9152-4422-a09e-eafb7fe72d2b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.700955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.701584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.702528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.702711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.703469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:48.703776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.705240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.732750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 386/764] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:16:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:48.734543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:48.806027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458036-dc14-4379-8b56-5687ce9315d8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.829189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:48.844775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c862cdf-e535-4f3a-b786-4d4f895fc6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.845438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c862cdf-e535-4f3a-b786-4d4f895fc6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.845724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c862cdf-e535-4f3a-b786-4d4f895fc6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.846005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c862cdf-e535-4f3a-b786-4d4f895fc6e9 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.846702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c862cdf-e535-4f3a-b786-4d4f895fc6e9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:48.846895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce8ac3-46b5-4e39-a46b-c745b8274829 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:48.847093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c862cdf-e535-4f3a-b786-4d4f895fc6e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.848370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c862cdf-e535-4f3a-b786-4d4f895fc6e9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.853440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 379/765] 10.222.0.128 () {56 vars in 1129 bytes} [Tue Jul 26 15:16:48 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:48.854680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8c20-443b-405b-8821-e7bf6334f312 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.854977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8c20-443b-405b-8821-e7bf6334f312 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.855256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8c20-443b-405b-8821-e7bf6334f312 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.856757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 380/766] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:48.860723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8c20-443b-405b-8821-e7bf6334f312 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:48.882852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:48.892961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4f1f8-96c1-40aa-8086-3d53bee60370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:48.893684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4f1f8-96c1-40aa-8086-3d53bee60370 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.894146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4f1f8-96c1-40aa-8086-3d53bee60370 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.894561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4f1f8-96c1-40aa-8086-3d53bee60370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.907527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4f1f8-96c1-40aa-8086-3d53bee60370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:48.909158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4f1f8-96c1-40aa-8086-3d53bee60370 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-19b4f1f8-96c1-40aa-8086-3d53bee60370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:48.929468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:48.931571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:48.933820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:48.933820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:48.933820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:48.933820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:48.940499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 387/767] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:48.940499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea3ce2-6e13-4aac-a13c-2900b7def21f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:48.940499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea3ce2-6e13-4aac-a13c-2900b7def21f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:48.940499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea3ce2-6e13-4aac-a13c-2900b7def21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:48.957545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a3ea3ce2-6e13-4aac-a13c-2900b7def21f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:48.958849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a3ea3ce2-6e13-4aac-a13c-2900b7def21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:48.977274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8c20-443b-405b-8821-e7bf6334f312 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.017452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.018131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.018430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.018711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.019551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:49.020109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.021372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.027498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 381/768] 10.222.0.128 () {56 vars in 1129 bytes} [Tue Jul 26 15:16:48 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:49.028376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.034852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 388/769] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:48 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:49.035813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea3ce2-6e13-4aac-a13c-2900b7def21f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.123999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.136175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.157184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5ef4-d7c8-4992-84ac-67a93b516d47 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.162956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea3ce2-6e13-4aac-a13c-2900b7def21f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.196319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.197192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.197548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.197975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.198754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:49.199133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.200625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.203681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.204322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.204743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.205071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.205851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] RBAC: Adding query filter params (interface=public, service_id=240be1ad04ad47dbb9aaf23930d3c37c, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:49.206217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.207742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.215192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 382/770] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:49.216442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.221357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 389/771] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/endpoints?service_id=240be1ad04ad47dbb9aaf23930d3c37c&interface=public®ion_id=RegionOne => generated 195 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:49.222863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.333148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374b249-fa83-411d-86a0-ad856107e2b9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.339105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e45d02-9f57-422b-9cf4-8112de02a78c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.372188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.376712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.377379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.377832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.378335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.378509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.379067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.379325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.379690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.379972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.380581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.388095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.390489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:49.392030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:49.394316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:49.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 383/772] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:49.398673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.406387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:49.407343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:49.409353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:49.411630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:49.419719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 390/773] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:49.419719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.518619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50b8c5-93b9-4ddc-b340-a2fc291024f2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.530280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a308351-f45c-41c3-b6f5-0166f2860305 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.555701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.556442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.556762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.557040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.557704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.559494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.565744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 384/774] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/services => generated 776 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:49.566583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.568321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d6454-5e87-4fa8-990e-2e48c60ea6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.568998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d6454-5e87-4fa8-990e-2e48c60ea6ef None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.569286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d6454-5e87-4fa8-990e-2e48c60ea6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.569566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d6454-5e87-4fa8-990e-2e48c60ea6ef None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.570268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d6454-5e87-4fa8-990e-2e48c60ea6ef None admin] RBAC: Adding query filter params (name=network) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:49.570580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d6454-5e87-4fa8-990e-2e48c60ea6ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.572014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d6454-5e87-4fa8-990e-2e48c60ea6ef None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.586984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 391/775] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/services?name=network => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:49.588504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebce05-1867-4424-a7b8-5d6ac8fff631 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.588804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebce05-1867-4424-a7b8-5d6ac8fff631 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.589087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebce05-1867-4424-a7b8-5d6ac8fff631 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.591455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 392/776] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:49.592586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebce05-1867-4424-a7b8-5d6ac8fff631 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.663553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.680326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.724321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0073d-3aa8-44c9-a638-9c7fedb3c96e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.734715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebce05-1867-4424-a7b8-5d6ac8fff631 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.766658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9d5b9-a7c6-4abf-8bdf-cbc95c26be47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.767516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9d5b9-a7c6-4abf-8bdf-cbc95c26be47 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.767821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9d5b9-a7c6-4abf-8bdf-cbc95c26be47 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.768177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9d5b9-a7c6-4abf-8bdf-cbc95c26be47 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.769110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9d5b9-a7c6-4abf-8bdf-cbc95c26be47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:49.769431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9d5b9-a7c6-4abf-8bdf-cbc95c26be47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.770879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9d5b9-a7c6-4abf-8bdf-cbc95c26be47 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.782459 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.783563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.784024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.784797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.785737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] RBAC: Adding query filter params (type=network) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:49.786228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.791525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 385/777] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:49.791525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1ef4-724b-443e-9fdc-6ec215830601 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.791525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1ef4-724b-443e-9fdc-6ec215830601 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.791525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1ef4-724b-443e-9fdc-6ec215830601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.793139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:49.804671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 393/778] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/services?type=network => generated 352 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:49.806502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.813063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-87cf1ef4-724b-443e-9fdc-6ec215830601 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:49.815145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-87cf1ef4-724b-443e-9fdc-6ec215830601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:49.903628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 386/779] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:49 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:49.903628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1ef4-724b-443e-9fdc-6ec215830601 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:49.946124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deecab-e37a-4c9e-a3e6-11c340a7ae1b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:49.971936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.988531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:49.996386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:49.996386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:49.996386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/92e40a33db1a41d59d9a7a3bef5f3ece/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:49.996386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=92e40a33db1a41d59d9a7a3bef5f3ece, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:49.996729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:49.996809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:50.042505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1ef4-724b-443e-9fdc-6ec215830601 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:50.087431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 394/780] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:49 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/92e40a33db1a41d59d9a7a3bef5f3ece/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:50.087431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:50.098611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:50.099365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:50.099690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:50.101275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:50.102146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] RBAC: Adding query filter params (interface=public, service_id=844a280836484855854b430cebfcdb06, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:50.104088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:50.107675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:50.132017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 387/781] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:16:49 2022] GET /identity/v3/endpoints?service_id=844a280836484855854b430cebfcdb06&interface=public®ion_id=RegionOne => generated 195 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:50.133777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:50.305897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a17c-4fd4-46d2-b113-484871a32086 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:50.322177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58129-18db-4066-b8e0-3dadc3a99e4d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:50.343593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:50.357524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:50.359302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:50.359588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:50.359863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:50.360580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:50.362336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:50.372624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 388/782] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:16:50 2022] GET /identity/v3/services => generated 776 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:50.376073 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf293c-4d9e-41e4-80ab-5c1f9d6c680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:50.376793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf293c-4d9e-41e4-80ab-5c1f9d6c680f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:50.377078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf293c-4d9e-41e4-80ab-5c1f9d6c680f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:50.377359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf293c-4d9e-41e4-80ab-5c1f9d6c680f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:50.390666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf293c-4d9e-41e4-80ab-5c1f9d6c680f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-19cf293c-4d9e-41e4-80ab-5c1f9d6c680f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:50.396258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:50.397336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:50.398175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 395/783] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:50.404387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:50.563210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12061f-692a-4622-aafe-370968f56ae8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:50.606199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:50.606825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:50.607108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:50.607428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:50.608211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:50.608542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:50.610083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:50.619978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 389/784] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:50.624131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7e40-bf53-4769-82a9-504bb20c28d3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:50.624623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7e40-bf53-4769-82a9-504bb20c28d3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:50.624884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7e40-bf53-4769-82a9-504bb20c28d3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:50.626636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 396/785] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:50.627261 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:50.632219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ef21-4cd5-49b6-87bc-f9313875323c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:50.632912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ef21-4cd5-49b6-87bc-f9313875323c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:50.633194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ef21-4cd5-49b6-87bc-f9313875323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:50.656742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e089ef21-4cd5-49b6-87bc-f9313875323c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:50.658191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e089ef21-4cd5-49b6-87bc-f9313875323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:50.686552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:50.712739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:50.770753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 397/786] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:50 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:16:50.771552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb23006-3368-4062-82a4-adb02a440249 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:50.828415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089ef21-4cd5-49b6-87bc-f9313875323c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:50.834893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:50.835592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:50.835886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:50.836170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:50.850213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:50.852110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:50.870326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:50.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:50.872172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:50.872172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:50.872172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:50.872172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:50.872172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 390/787] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:50.877855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.020296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089ef21-4cd5-49b6-87bc-f9313875323c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.057981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.058636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.059087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.059375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.071941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:51.077899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:51.078989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:51.080442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 398/788] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:51.085534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.090292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0b23f-3460-414c-b639-f649a8418f85 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.092150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:51.120092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:51.161183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.162199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.162675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.163124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.164006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:51.164510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:51.166074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:51.195592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 391/789] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:51.199418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229776e5-e637-49ff-9abb-713735e5385e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.203434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.255852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.256514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.256806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.257084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.257825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:51.258135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:51.259523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:51.291248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 399/790] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:51 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:51.305171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.414080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589953cd-5db9-4cbc-a395-6a3354c0d251 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.425261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:51.435537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ef74-da00-4238-ab7b-e10389f2c205 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.472563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.473230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.473518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.473800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.490790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:51.492473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:51.495596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.496261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.496576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.496857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:51.516652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:51.517777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:16:51.519078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:51.522986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 392/791] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:16:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:51.523771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:51.525395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:51.526715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:51.556023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:51.558662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:51.561834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:51.561834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:51.561834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:51.561834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:51.561834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 400/792] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:51.569836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.587906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:51.741432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade4f1e-77d1-48fa-9edc-eb55d2d18727 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.751757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f333f-1ea4-4eff-a610-dee7b7829362 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.787667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.788490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.788893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.789278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.790086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:51.790503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:51.791989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:51.805714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 401/793] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:51.812028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.815939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:51.822788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.823604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.824024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.824441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.825425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:51.825880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:51.828486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:51.851747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 393/794] 10.222.0.128 () {56 vars in 1154 bytes} [Tue Jul 26 15:16:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:51.854452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:51.939430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aba512-e02a-4263-990a-778d31b72a39 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:51.960838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:51.979309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:51.980230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:51.980619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:51.981016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:51.990832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:51.993007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:52.009406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:52.010426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:52.011358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:52.012455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a345948-aa19-4198-b4b1-416b6b0a9800 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.012574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 402/795] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:52.017844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.095051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:52.105734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881f2ae-fd29-4f96-8972-d8f399885731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.106707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f2ae-fd29-4f96-8972-d8f399885731 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.107224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f2ae-fd29-4f96-8972-d8f399885731 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.107828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f2ae-fd29-4f96-8972-d8f399885731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.127603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d092caf-9cbd-42f5-b2ed-261d19a9b15b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.129253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881f2ae-fd29-4f96-8972-d8f399885731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.145430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881f2ae-fd29-4f96-8972-d8f399885731 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5881f2ae-fd29-4f96-8972-d8f399885731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:52.149264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:52.150127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:52.151023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:52.151023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:52.151023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:52.151023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:52.156958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 394/796] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:52.161642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b3852-7328-40c2-bb11-99980b90a149 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.162609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b3852-7328-40c2-bb11-99980b90a149 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.163266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b3852-7328-40c2-bb11-99980b90a149 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.165327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 395/797] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:52 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:52.166320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b3852-7328-40c2-bb11-99980b90a149 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.178066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15892001-a3cb-47e5-a66a-7c31b73688cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.178855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15892001-a3cb-47e5-a66a-7c31b73688cb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.179260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15892001-a3cb-47e5-a66a-7c31b73688cb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.179610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15892001-a3cb-47e5-a66a-7c31b73688cb None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.180967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15892001-a3cb-47e5-a66a-7c31b73688cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:52.181636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15892001-a3cb-47e5-a66a-7c31b73688cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.183950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15892001-a3cb-47e5-a66a-7c31b73688cb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.215805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 403/798] 10.222.0.128 () {56 vars in 1140 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:52.221997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378fa58-d1fd-4536-9060-cbe10ec11656 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.222366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378fa58-d1fd-4536-9060-cbe10ec11656 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.222713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378fa58-d1fd-4536-9060-cbe10ec11656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.250513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6378fa58-d1fd-4536-9060-cbe10ec11656 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:52.250513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6378fa58-d1fd-4536-9060-cbe10ec11656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:52.302372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b3852-7328-40c2-bb11-99980b90a149 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.340245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:52.352498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 404/799] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:52 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:52.352498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378fa58-d1fd-4536-9060-cbe10ec11656 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.358312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.358312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.358312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.358312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.358784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:52.358784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.359722 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.381054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 396/800] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:52.382481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.422335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:52.491613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378fa58-d1fd-4536-9060-cbe10ec11656 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.523525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc398cd7-09c3-4e15-bf8a-0d8062a95488 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.535451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.536248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.536998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.536998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.549485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.551332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.569174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:52.569413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:52.570253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:52.571176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:52.571176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:52.571176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:52.571176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:52.572394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.572870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.573327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.574206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.575434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 405/801] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:52.576130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.577059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:52.588107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:52.589156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 15:16:52.590001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:52.598603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 397/802] 10.222.0.128 () {56 vars in 1134 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:52.598603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.664247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:52.706909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ce149-1d48-4a5e-aec8-149687e63349 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.709539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:52.715567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69796e-4d68-467c-b78e-6a1d3af37be7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.743240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e412-b0d5-4e4b-9c4a-183b1415fe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.744167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e412-b0d5-4e4b-9c4a-183b1415fe1e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.744513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e412-b0d5-4e4b-9c4a-183b1415fe1e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.745237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e412-b0d5-4e4b-9c4a-183b1415fe1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.752714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.753350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.753637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.753916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.754242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989e412-b0d5-4e4b-9c4a-183b1415fe1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.754710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:52.754929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.755725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989e412-b0d5-4e4b-9c4a-183b1415fe1e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.756514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2989e412-b0d5-4e4b-9c4a-183b1415fe1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:52.770141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:52.775947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:52.776929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:52.777362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 398/803] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:52.777362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.777491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 406/804] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:52.777491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1d03-a146-42bb-91ff-5a4c6c4efe80 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.778085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1d03-a146-42bb-91ff-5a4c6c4efe80 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.778085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1d03-a146-42bb-91ff-5a4c6c4efe80 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.783726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 407/805] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:52 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:52.784576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb1d03-a146-42bb-91ff-5a4c6c4efe80 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.885904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce84626-5a75-499e-85aa-572aa08bb7bc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.897648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb1d03-a146-42bb-91ff-5a4c6c4efe80 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:52.904385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:52.923533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.924173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.924484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.924773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.925473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:52.925786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.927381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.937661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 399/806] 10.222.0.128 () {56 vars in 1129 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:52.937661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:52.941435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe94aac-8918-4198-b775-b7f8dc890574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:52.942359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94aac-8918-4198-b775-b7f8dc890574 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.942657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94aac-8918-4198-b775-b7f8dc890574 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.942919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94aac-8918-4198-b775-b7f8dc890574 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.950468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe94aac-8918-4198-b775-b7f8dc890574 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:52.952457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe94aac-8918-4198-b775-b7f8dc890574 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-bfe94aac-8918-4198-b775-b7f8dc890574 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:52.965041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:52.965957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 408/807] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6310ceb-7fc3-45d6-97a1-82e461316035 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:52.969458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6310ceb-7fc3-45d6-97a1-82e461316035 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:52.970872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6310ceb-7fc3-45d6-97a1-82e461316035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:52.988490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f6310ceb-7fc3-45d6-97a1-82e461316035 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:52.989685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f6310ceb-7fc3-45d6-97a1-82e461316035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:53.007885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:53.061276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e6a31-77d5-455f-836b-3971bb9bdb96 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.061870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 409/808] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:52 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:53.062799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6310ceb-7fc3-45d6-97a1-82e461316035 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.098123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.098123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.098123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.098555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.099087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:53.099404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:53.100883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:53.112885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 400/809] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:53.113718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.158651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:53.170121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6310ceb-7fc3-45d6-97a1-82e461316035 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.206107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2a907-6c7e-4f77-9d26-5c6db58b2e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.206736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a907-6c7e-4f77-9d26-5c6db58b2e5b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.207023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a907-6c7e-4f77-9d26-5c6db58b2e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.207300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a907-6c7e-4f77-9d26-5c6db58b2e5b None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.207986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a907-6c7e-4f77-9d26-5c6db58b2e5b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:53.208287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a907-6c7e-4f77-9d26-5c6db58b2e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:53.209715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a907-6c7e-4f77-9d26-5c6db58b2e5b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:53.214765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 410/810] 10.222.0.128 () {56 vars in 1129 bytes} [Tue Jul 26 15:16:53 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:53.215951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4987639-c878-45b4-aa26-d3d5cfb55b9f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.216238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4987639-c878-45b4-aa26-d3d5cfb55b9f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.216542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4987639-c878-45b4-aa26-d3d5cfb55b9f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.218334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 411/811] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:53.219061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4987639-c878-45b4-aa26-d3d5cfb55b9f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.225331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200e08-d80c-4169-8e0c-c90e057d8498 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.238490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:53.266758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.267381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.267663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.267937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.268723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:53.269032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:53.270452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:53.284609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 401/812] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:53.285592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.336068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4987639-c878-45b4-aa26-d3d5cfb55b9f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.371957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.372635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.372925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.374138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.385219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:53.390956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:53.391235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:53.392165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:53.393160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 412/813] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:53.393857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.398377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19587f-51b8-4dfd-abf7-5226205b8c8c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.436295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cc45a-9795-46fe-a8fa-357d4aa437f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.437010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc45a-9795-46fe-a8fa-357d4aa437f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.437293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc45a-9795-46fe-a8fa-357d4aa437f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.437571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc45a-9795-46fe-a8fa-357d4aa437f9 None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/f35a6f8e44fa42dca9ab2ea540ba57ac/roles/224e0c1f1992436bbcf1ba18913cb81b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.452531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cc45a-9795-46fe-a8fa-357d4aa437f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=f35a6f8e44fa42dca9ab2ea540ba57ac, role_id=224e0c1f1992436bbcf1ba18913cb81b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:53.453983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cc45a-9795-46fe-a8fa-357d4aa437f9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:53.481951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:53.508372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e1499-a8df-442c-9ab8-0c6e2b9c4b95 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.542550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.543194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.543475 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.543752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.544495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:53.547098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:53.550374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 402/814] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:53 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/f35a6f8e44fa42dca9ab2ea540ba57ac/roles/224e0c1f1992436bbcf1ba18913cb81b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:53.552350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee9f34-3b8c-4560-966d-93afafea714f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.552837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee9f34-3b8c-4560-966d-93afafea714f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.553436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee9f34-3b8c-4560-966d-93afafea714f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.574168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-27ee9f34-3b8c-4560-966d-93afafea714f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:53.575427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-27ee9f34-3b8c-4560-966d-93afafea714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:53.630778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 413/815] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:16:53 2022] POST /identity/v3/endpoints => generated 348 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:53.631580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.690265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 403/816] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:53 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:53.691726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee9f34-3b8c-4560-966d-93afafea714f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.787981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eed74e-082f-4b5a-b2a3-839d7993662b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.793407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:53.827450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee9f34-3b8c-4560-966d-93afafea714f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:53.830186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.831087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.831369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.831648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/97295dc4a04143fe879bef062964e4e0/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.833713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=97295dc4a04143fe879bef062964e4e0, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:53.835253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:53.837796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:53.872914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:53.872914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:53.872914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:53.873912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:53.874470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:53.874776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:53.876306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:53.890799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 404/817] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:53 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:53.890799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:53.923229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 414/818] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:16:53 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/97295dc4a04143fe879bef062964e4e0/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:16:53.928503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:54.095092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474122cc-fa68-41bb-b96f-49f29a6f7624 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:54.141757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ef4e-5dbb-4455-9835-147103b0c5dc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:54.158951 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:54.159782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:54.160080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:54.160621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:54.161279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:54.162734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:54.177777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:54.178848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 15:16:54.179684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:54.180439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 405/819] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:53 2022] GET /identity/v3/services/network => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:54.188350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:54.193483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:54.197581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:54.198241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:54.198523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:54.198852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:54.218866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:54.220887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:54.237102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:54.238162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:54.239076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:54.239076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:54.239076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:54.239076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:54.241350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 415/820] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:54.248154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:54.306431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:54.360811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f7539-36ee-43b6-8337-5f36ed340423 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:54.397699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:54.398978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:54.399481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:54.399961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:54.400912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:54.401426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:54.403008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:54.417353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 416/821] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:54.428515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:54.448734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fe620-e13b-400e-9fe7-feaf84314da9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:54.518452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:54.519440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:54.519725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:54.520123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:54.520973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] RBAC: Adding query filter params (name=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:54.521476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:54.525239 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:54.539656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 406/822] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:16:54 2022] GET /identity/v3/services?name=network => generated 125 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:54.550243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:54.639817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84a3ec-c7c8-4bae-8639-0481231501ee None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:54.655922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:54.679731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:54.710040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:54.711089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:54.711670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:54.712156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:54.728210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:54.730114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:54.750051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8688d64-3869-415e-87fe-b89b9a668b91 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:54.761120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:54.762098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:16:54.763032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:54.768825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 417/823] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:16:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:54.771254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:54.826924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:54.828239 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:54.828760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:54.829235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:54.830126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] RBAC: Adding query filter params (type=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:54.831795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:54.833418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:54.844646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 407/824] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:16:54 2022] GET /identity/v3/services?type=network => generated 352 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:54.850856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:54.970533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74f1c5-7e61-48a4-8943-945fe97cc9f6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:55.012498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:55.012498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:55.012498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:55.012498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:55.012907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:55.012907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:55.012907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:55.033632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 418/825] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:16:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:55.042092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:55.071727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:55.090554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698d52-a0b6-4cf4-8c99-ad5d187a9cbe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:55.167427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07d15-f298-47d2-9725-18240a5c426b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:55.174390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:55.196277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:55.197232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:55.197660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:55.198081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:55.199059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:55.200626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:55.219324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:55.220360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:55.223821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:55.223821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:55.303922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:55.305855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:55.334906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:55.364778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 408/826] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:16:54 2022] POST /identity/v3/endpoints => generated 329 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:16:55.440945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 419/827] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:55 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:16:55.446767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:55.620297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126451ef-9345-4ade-8e15-b266ae19d062 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:55.659816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454785-eb59-4ca0-a934-ccd2e9f9db3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:55.660930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454785-eb59-4ca0-a934-ccd2e9f9db3b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:55.661389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454785-eb59-4ca0-a934-ccd2e9f9db3b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:55.661816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454785-eb59-4ca0-a934-ccd2e9f9db3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:55.675344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e454785-eb59-4ca0-a934-ccd2e9f9db3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:55.677187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e454785-eb59-4ca0-a934-ccd2e9f9db3b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3e454785-eb59-4ca0-a934-ccd2e9f9db3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:55.694175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:55.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:55.696381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:55.696381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:55.696381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:55.696381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:55.698222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 409/828] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:55.705875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:55.827218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b96f-e640-46d8-a54f-0ddd75197726 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:55.827805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a67b3-2c19-46db-ab18-42cf5c632126 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:55.828259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a67b3-2c19-46db-ab18-42cf5c632126 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:55.828733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a67b3-2c19-46db-ab18-42cf5c632126 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:55.830503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 410/829] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:16:55.837223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdde7f1-fd91-44e2-af43-424741527766 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:55.837877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdde7f1-fd91-44e2-af43-424741527766 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:55.838414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdde7f1-fd91-44e2-af43-424741527766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:55.858744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9fdde7f1-fd91-44e2-af43-424741527766 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:55.860563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:55.861982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9fdde7f1-fd91-44e2-af43-424741527766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:55.878865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:55.878865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:55.878865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:55.878865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:55.879933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:55.880486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:55.882096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:55.896456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 420/830] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:55.900639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:55.902084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:55.959223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 411/831] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:55 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:55.987274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdde7f1-fd91-44e2-af43-424741527766 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.030413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3760be-7256-4206-a260-400f867f7dfa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.077689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.078735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.079341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.079925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.090063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:56.093018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:56.100712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdde7f1-fd91-44e2-af43-424741527766 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:56.114182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:56.115425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:56.116322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:56.117923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 421/832] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:56.125761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.151371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.151969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.152250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.152555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.165156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:56.170938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:56.171885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:16:56.172762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:56.176872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 412/833] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:16:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:56.178460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.187070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:56.218067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:56.255408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44736e50-af59-40fb-9d98-1a4135407301 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.298877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a654571-809b-435c-94f1-f989a3e55c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.299530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a654571-809b-435c-94f1-f989a3e55c3a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.299807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a654571-809b-435c-94f1-f989a3e55c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.300083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a654571-809b-435c-94f1-f989a3e55c3a None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.300893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a654571-809b-435c-94f1-f989a3e55c3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:56.301207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a654571-809b-435c-94f1-f989a3e55c3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:56.302633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a654571-809b-435c-94f1-f989a3e55c3a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:56.304455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f0cee-cd55-422a-be4c-df6ae5a0469c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.321731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 422/834] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:56.323212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca4d1-d866-49a4-9fbc-fd99d872a66c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.323491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca4d1-d866-49a4-9fbc-fd99d872a66c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.323780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca4d1-d866-49a4-9fbc-fd99d872a66c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.325163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 423/835] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:56.325986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990713b8-6a9f-44fe-9a8e-3191cb452de2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.326875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990713b8-6a9f-44fe-9a8e-3191cb452de2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.327145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990713b8-6a9f-44fe-9a8e-3191cb452de2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.332527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 424/836] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:56.332527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990713b8-6a9f-44fe-9a8e-3191cb452de2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.348840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49e653-8554-425f-abc9-41662443a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.349474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49e653-8554-425f-abc9-41662443a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.349755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49e653-8554-425f-abc9-41662443a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.350030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49e653-8554-425f-abc9-41662443a8c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.350758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49e653-8554-425f-abc9-41662443a8c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:56.351068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49e653-8554-425f-abc9-41662443a8c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:56.352444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49e653-8554-425f-abc9-41662443a8c8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:56.361009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 413/837] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:56.362329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e3330-78f6-430c-9265-8de84ed93d5f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.362614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e3330-78f6-430c-9265-8de84ed93d5f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.362891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e3330-78f6-430c-9265-8de84ed93d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.381646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7c7e3330-78f6-430c-9265-8de84ed93d5f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:56.382858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7c7e3330-78f6-430c-9265-8de84ed93d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:56.445827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990713b8-6a9f-44fe-9a8e-3191cb452de2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.448054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:56.464315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 414/838] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:56 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:56.465647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d282856-f12b-4716-ba66-e033c57e0136 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.465937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d282856-f12b-4716-ba66-e033c57e0136 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.466217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d282856-f12b-4716-ba66-e033c57e0136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.484079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.485303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.485842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.485958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.486742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=92e40a33db1a41d59d9a7a3bef5f3ece) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:56.487061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:56.488533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:56.491833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6d282856-f12b-4716-ba66-e033c57e0136 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:56.493109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6d282856-f12b-4716-ba66-e033c57e0136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:56.496247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 425/839] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/role_assignments?user.id=92e40a33db1a41d59d9a7a3bef5f3ece&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:56.497257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.510007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:56.590929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 415/840] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:56 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:56.592033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d282856-f12b-4716-ba66-e033c57e0136 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.634413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb105c2-e320-4ef8-8332-ee5701ec9685 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.675268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.675924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.676206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.676501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.691940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:56.694010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:56.723353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:56.724308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:56.728235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:56.728235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:56.728235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:56.728235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:56.728235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 426/841] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:56.728235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.737709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d282856-f12b-4716-ba66-e033c57e0136 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.754682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:56.777396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.778399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.778837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.779295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.780513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:56.782152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:56.793228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:56.794599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:16:56.795966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:56.797017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 416/842] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:56.797973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.813538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:56.860520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01a049-18fd-4553-ae0d-a9136f8d4efd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.903291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.904221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.905273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.905273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.919649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:16:56.924318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff279cd-d7bf-4301-b4ae-aede6788d6ea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:16:56.926623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:56.927563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:56.928440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 427/843] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:56.930478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:56.971057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:56.971726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:56.972014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:56.972298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:56.973047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:56.973414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:56.974748 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:56.992312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 417/844] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:56.992312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.033147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:57.055761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5c895-5303-42db-9668-9780fcda1d95 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.072448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:57.096394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.097213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.097934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.098259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.098959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:57.099269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.100714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.106821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 428/845] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:57.107943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.117008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26049b2-0f43-4716-9999-21c854b3eeed None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.156068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.157015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.157015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.157015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.160274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:57.160274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.160274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.174410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 418/846] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:16:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:57.175361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.228983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0b59-1ca9-43a0-bab8-75fa73349ed2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.267005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.268690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.269178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.269671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.282071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.283651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.295433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032214c-8cb2-4398-a1f3-f01946528029 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:57.296071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:57.297126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:16:57.298301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:57.302337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 429/847] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:57.303096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.308026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:57.335540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:57.342620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.343551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.343985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.344438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.345573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:57.346114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.348269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.364496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 419/848] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:57.364496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.447733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fb354-6ed9-4065-874f-fa0f838ec65a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.503730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.505426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.505956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.506963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.525014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.527685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.534877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd27eae-f209-4fb0-a5f1-e46f11e63404 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:57.548300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:57.549415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:57.550318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:57.550318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:57.550318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:57.550318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:57.550318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 430/849] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:57.555093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.580531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.581212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.581497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.581839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.582742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:57.583049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.584561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.608576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 420/850] 10.222.0.128 () {56 vars in 1154 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:57.613714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.625049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:57.684446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231758d-bb0e-4f0b-930a-00fa1c409c5f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.700024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:57.729478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.730136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.730424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.730707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.731435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:57.731749 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.734758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.749662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 431/851] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:16:57.756014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.771439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9fd50-3dd7-4199-a150-6a122acae273 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.816746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.817385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.817673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.818100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.889564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1045d-7eb7-44a7-9489-4aa4873b54aa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:57.913093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.913942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:57.916297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.940432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:57.941488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:57.941875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:57.942252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:57.954967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:57.957160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:16:57.975308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:57.976350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 15:16:57.977342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:57.977879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 432/852] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:16:57.985533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:57.990804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:58.043285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 421/853] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 15:16:58.045461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:58.137730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668c26d-ee81-41a5-9c0f-23fb7a06c558 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:58.183516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:58.184586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.185136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.185552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.187081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:58.187848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:58.189677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:58.190741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f94be-240a-455b-84eb-21928fb3e8c8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:58.215870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 433/854] 10.222.0.128 () {56 vars in 1140 bytes} [Tue Jul 26 15:16:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:58.222915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:58.238492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:58.239488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.240046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.240328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.255332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:58.257575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:58.262797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:58.278886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:58.281279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:58.282492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:58.282492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:58.282492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:58.282492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:58.282492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 422/855] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:58.289816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:58.302021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:58.357957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991f4ca-786b-4d6a-8655-1b0efea6cf53 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:58.403211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:58.404023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.404309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.404727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.413724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d12c2-aa85-4a83-a321-2d140f25c1f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:58.460284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:58.461042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.461328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.461608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.462319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:58.462635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:58.464448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:58.487517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 423/856] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:58.489489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:58.512099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:58.514128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:58.562372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:58.572197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:58.634722 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041812b-fcc2-4138-94df-5efafa697886 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:58.656022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 434/857] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:16:58.661993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:58.681275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bd5e6-352a-46b3-a613-c4e1b166b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:58.682297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bd5e6-352a-46b3-a613-c4e1b166b72f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.682651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bd5e6-352a-46b3-a613-c4e1b166b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.682996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bd5e6-352a-46b3-a613-c4e1b166b72f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.693816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bd5e6-352a-46b3-a613-c4e1b166b72f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:58.695287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bd5e6-352a-46b3-a613-c4e1b166b72f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c78bd5e6-352a-46b3-a613-c4e1b166b72f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:58.715051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:58.716018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:58.717700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 424/858] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:58.719063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206f7d4-f278-44f1-8f8a-d9c0954e42c9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.719063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206f7d4-f278-44f1-8f8a-d9c0954e42c9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.719063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206f7d4-f278-44f1-8f8a-d9c0954e42c9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.720228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 425/859] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:58.724877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206f7d4-f278-44f1-8f8a-d9c0954e42c9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:58.787751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e7f43-1bf4-49a3-839e-266ab8754d50 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:58.829129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f28dbe-8bd9-4de6-95bd-8ccd4b2ee4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:58.829905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28dbe-8bd9-4de6-95bd-8ccd4b2ee4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.830138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28dbe-8bd9-4de6-95bd-8ccd4b2ee4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.830393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28dbe-8bd9-4de6-95bd-8ccd4b2ee4d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.832655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:58.834817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206f7d4-f278-44f1-8f8a-d9c0954e42c9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:58.843502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f28dbe-8bd9-4de6-95bd-8ccd4b2ee4d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:58.845145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f28dbe-8bd9-4de6-95bd-8ccd4b2ee4d9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-75f28dbe-8bd9-4de6-95bd-8ccd4b2ee4d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:16:58.861827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:58.862852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:58.863739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:16:58.863739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:16:58.863739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:16:58.863739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:58.864562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:58.864908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 435/860] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:58.865184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801a54-6712-43dc-a66a-532f3d786d1d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.865465 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801a54-6712-43dc-a66a-532f3d786d1d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.865752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801a54-6712-43dc-a66a-532f3d786d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.877662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:58.878287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:58.878572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:58.878854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:58.879619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:58.881472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:58.881472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:58.885831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a8801a54-6712-43dc-a66a-532f3d786d1d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:58.887054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a8801a54-6712-43dc-a66a-532f3d786d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:58.894883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 426/861] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:58.896110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:58.965253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 436/862] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:58 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:58.966104 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8801a54-6712-43dc-a66a-532f3d786d1d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.028766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941c985-2b3a-4efe-b603-b58cdf14e1e8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.068931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.069719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.069826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.070155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.070787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:59.071109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.072500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.084451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 427/863] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:59.087955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.108683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8801a54-6712-43dc-a66a-532f3d786d1d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.134253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:59.144523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:59.154952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.155586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.155863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.156147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.157025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=f35a6f8e44fa42dca9ab2ea540ba57ac) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:59.157324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.158774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.176504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 437/864] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:58 2022] GET /identity/v3/role_assignments?user.id=f35a6f8e44fa42dca9ab2ea540ba57ac&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 610 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:59.176504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.231508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddeff9-6919-4342-afd1-db7944f7b5c7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.273144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.273844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.274132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.274413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.275069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.276530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:59.287607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:59.288952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:16:59.289867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:16:59.295201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 428/865] 10.222.0.128 () {56 vars in 1137 bytes} [Tue Jul 26 15:16:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:59.296271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.324237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf9549-1652-4139-b657-44421e409f39 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.374018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.374254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.374398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.374682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.382638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.384441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:16:59.396771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:16:59.397733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:16:59.398625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 438/866] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:16:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:16:59.403668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.428303 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfdd59-e9e9-4bbb-b2f7-94cf61c17fc9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.443045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:59.458661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:59.481544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.483170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.484351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.484351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.486874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:59.486874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.486874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.493645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 429/867] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:16:59 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:59.499649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.547892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51800c98-3775-406e-b311-2c2b24018810 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.590074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b792e8-c8cc-4dcf-90c2-f9cf2ff6db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.590698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b792e8-c8cc-4dcf-90c2-f9cf2ff6db84 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.590979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b792e8-c8cc-4dcf-90c2-f9cf2ff6db84 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.591253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b792e8-c8cc-4dcf-90c2-f9cf2ff6db84 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.593007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b792e8-c8cc-4dcf-90c2-f9cf2ff6db84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:59.593741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b792e8-c8cc-4dcf-90c2-f9cf2ff6db84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.595424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b792e8-c8cc-4dcf-90c2-f9cf2ff6db84 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.608353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 439/868] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:16:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:59.609934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95410a-6f92-496a-8cdf-86c4b0dc4b2f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.610364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95410a-6f92-496a-8cdf-86c4b0dc4b2f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.610779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95410a-6f92-496a-8cdf-86c4b0dc4b2f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.616366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 440/869] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:16:59 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:16:59.617407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95410a-6f92-496a-8cdf-86c4b0dc4b2f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.627472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167d1a-4a32-435d-aa57-f70531423207 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.674131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21fec8-921d-4c89-ae1d-d087afd19b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.674760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21fec8-921d-4c89-ae1d-d087afd19b49 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.675041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21fec8-921d-4c89-ae1d-d087afd19b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.675314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21fec8-921d-4c89-ae1d-d087afd19b49 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.676312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21fec8-921d-4c89-ae1d-d087afd19b49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:59.676312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21fec8-921d-4c89-ae1d-d087afd19b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.677736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21fec8-921d-4c89-ae1d-d087afd19b49 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.688509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 430/870] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:16:59 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:59.688509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320eabb-da01-416b-b7ec-b164f6fd42f3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.688509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320eabb-da01-416b-b7ec-b164f6fd42f3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.688509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320eabb-da01-416b-b7ec-b164f6fd42f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.710299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9320eabb-da01-416b-b7ec-b164f6fd42f3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:59.711842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9320eabb-da01-416b-b7ec-b164f6fd42f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:59.749474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:59.751624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:59.756983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95410a-6f92-496a-8cdf-86c4b0dc4b2f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.798583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 431/871] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:16:59 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:16:59.816697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b1822-8c36-4f1e-b951-03a70b895c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.817789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b1822-8c36-4f1e-b951-03a70b895c6e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:59.818243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b1822-8c36-4f1e-b951-03a70b895c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:59.818674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b1822-8c36-4f1e-b951-03a70b895c6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:59.819774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b1822-8c36-4f1e-b951-03a70b895c6e None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=97295dc4a04143fe879bef062964e4e0) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:16:59.820540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b1822-8c36-4f1e-b951-03a70b895c6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:59.822750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320eabb-da01-416b-b7ec-b164f6fd42f3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:59.827885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b1822-8c36-4f1e-b951-03a70b895c6e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:59.836571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 441/872] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:16:59 2022] GET /identity/v3/role_assignments?user.id=97295dc4a04143fe879bef062964e4e0&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:16:59.959275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320eabb-da01-416b-b7ec-b164f6fd42f3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:59.999244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2a16b-55a2-47d3-9be3-ec24b3828196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:59.999882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2a16b-55a2-47d3-9be3-ec24b3828196 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:00.000165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2a16b-55a2-47d3-9be3-ec24b3828196 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:00.000439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2a16b-55a2-47d3-9be3-ec24b3828196 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:00.001180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2a16b-55a2-47d3-9be3-ec24b3828196 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:00.002646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2a16b-55a2-47d3-9be3-ec24b3828196 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:00.085367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 432/873] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:16:59 2022] POST /identity/v3/services => generated 252 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:01.530002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312521a-be96-4e2e-904a-1965d99527f8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:01.531123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312521a-be96-4e2e-904a-1965d99527f8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:01.531643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312521a-be96-4e2e-904a-1965d99527f8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:01.533534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 442/874] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:01 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:01.539386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d0988-3582-4f47-8808-38fd4012cca3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:01.540285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d0988-3582-4f47-8808-38fd4012cca3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:01.540890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d0988-3582-4f47-8808-38fd4012cca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:01.561018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a73d0988-3582-4f47-8808-38fd4012cca3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:01.562610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a73d0988-3582-4f47-8808-38fd4012cca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:01.587893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:01.644587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 433/875] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:01 2022] POST /identity/v3/auth/tokens => generated 1254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:01.664503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312521a-be96-4e2e-904a-1965d99527f8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:01.778676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312521a-be96-4e2e-904a-1965d99527f8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:01.805083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8b0e-7b92-4c69-88e4-801ff5783203 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:01.805731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8b0e-7b92-4c69-88e4-801ff5783203 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:01.806212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8b0e-7b92-4c69-88e4-801ff5783203 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:01.808045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 434/876] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:01.814150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657ca9b-b5b6-4081-b0bc-b0609ba96f9b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:01.814747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657ca9b-b5b6-4081-b0bc-b0609ba96f9b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:01.815321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657ca9b-b5b6-4081-b0bc-b0609ba96f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:01.816979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:01.817637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:01.817923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:01.818205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:01.830369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:01.835362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:01.836321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:01.837213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:01.839190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2657ca9b-b5b6-4081-b0bc-b0609ba96f9b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:01.840657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2657ca9b-b5b6-4081-b0bc-b0609ba96f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:01.842145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 443/877] 10.222.0.128 () {56 vars in 1125 bytes} [Tue Jul 26 15:17:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:01.843580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:01.874341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:01.944604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 435/878] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:01 2022] POST /identity/v3/auth/tokens => generated 1254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:01.962002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c845206-712a-40ee-83b2-094fc1d528cb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:01.966046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e25b8-1607-4716-8005-356fff31d592 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:01.966842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e25b8-1607-4716-8005-356fff31d592 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:01.967131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e25b8-1607-4716-8005-356fff31d592 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:01.968643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 436/879] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:01.969321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e25b8-1607-4716-8005-356fff31d592 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.005813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4913de9-74c5-4b61-885e-a060429d9ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.006446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4913de9-74c5-4b61-885e-a060429d9ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.006735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4913de9-74c5-4b61-885e-a060429d9ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.007012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4913de9-74c5-4b61-885e-a060429d9ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.007737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4913de9-74c5-4b61-885e-a060429d9ee4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:02.008047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4913de9-74c5-4b61-885e-a060429d9ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.009450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4913de9-74c5-4b61-885e-a060429d9ee4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.018096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 444/880] 10.222.0.128 () {56 vars in 1120 bytes} [Tue Jul 26 15:17:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:02.019313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89b9d5-d4ab-4e28-9542-5ce650bdadad None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.019604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89b9d5-d4ab-4e28-9542-5ce650bdadad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.019930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89b9d5-d4ab-4e28-9542-5ce650bdadad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.038012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7f89b9d5-d4ab-4e28-9542-5ce650bdadad None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:02.039128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7f89b9d5-d4ab-4e28-9542-5ce650bdadad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:02.072362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.092507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e25b8-1607-4716-8005-356fff31d592 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.112066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 445/881] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:02 2022] POST /identity/v3/auth/tokens => generated 1254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:02.117182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89b9d5-d4ab-4e28-9542-5ce650bdadad None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.137229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.145157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.148785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.148785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.148785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.148785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.150638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.235151 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89b9d5-d4ab-4e28-9542-5ce650bdadad None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 437/882] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:17:01 2022] POST /identity/v3/services => generated 270 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:02.268755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.285035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb965e4-c9d5-4479-9680-5f9759b4d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.285710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb965e4-c9d5-4479-9680-5f9759b4d892 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.285994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb965e4-c9d5-4479-9680-5f9759b4d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.286270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb965e4-c9d5-4479-9680-5f9759b4d892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.298990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb965e4-c9d5-4479-9680-5f9759b4d892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.300293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb965e4-c9d5-4479-9680-5f9759b4d892 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fbb965e4-c9d5-4479-9680-5f9759b4d892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:02.318550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:02.328713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 446/883] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a745b0b-720b-48c6-a279-3ef41b0eec37 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a745b0b-720b-48c6-a279-3ef41b0eec37 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a745b0b-720b-48c6-a279-3ef41b0eec37 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.329612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 447/884] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:02 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:02.330156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a745b0b-720b-48c6-a279-3ef41b0eec37 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.401479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadff5e-bb5d-43d2-bccd-4234c4815ad4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.407457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.445897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e2a36-59bb-4243-9444-5154dbdb4ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.446566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e2a36-59bb-4243-9444-5154dbdb4ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.446862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e2a36-59bb-4243-9444-5154dbdb4ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.447143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e2a36-59bb-4243-9444-5154dbdb4ce9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.447791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e2a36-59bb-4243-9444-5154dbdb4ce9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.449220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e2a36-59bb-4243-9444-5154dbdb4ce9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.453309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a745b0b-720b-48c6-a279-3ef41b0eec37 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-555e2a36-59bb-4243-9444-5154dbdb4ce9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:02.459032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:02.460076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:02.461073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 438/885] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:02.462959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c21c-3147-48a3-8206-f07230fdce03 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.462959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c21c-3147-48a3-8206-f07230fdce03 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.462959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c21c-3147-48a3-8206-f07230fdce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.486330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2244c21c-3147-48a3-8206-f07230fdce03 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:02.487531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2244c21c-3147-48a3-8206-f07230fdce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:02.490114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.496772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.497290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.498115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.498544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.499269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:02.499579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.501010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.515236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 448/886] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:02.516101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.582888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 439/887] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:02 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:02.584370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244c21c-3147-48a3-8206-f07230fdce03 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.641054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a8000-e206-4c68-ab33-ea4d3cdf0f85 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.667423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.679943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.680697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.680983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.681261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.681993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:02.682305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.683762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.694793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 449/888] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:02.694793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.702215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244c21c-3147-48a3-8206-f07230fdce03 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.741977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.742635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.742966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.743250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.752885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.754346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.756742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:17:02.768228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:02.773568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:02.774584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 440/889] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:02.777305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.815905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c205252-c5af-4a8e-a2ef-fa89d3cf326f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.854729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.855295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.855577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.855851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.856546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.858788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:02.868154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:02.869226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:02.870079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 450/890] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:02.871585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.890534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8e437-b835-4ffd-acd1-c2d229f7b36a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.928852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.929478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.929761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.930036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.930718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:02.931022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.932452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.935033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.948496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 441/891] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:02.948496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.014225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:03.020159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b08c7c-146e-44f2-9627-269fb9c29394 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.065906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a59b7-912f-45f2-ab87-d18c81b416a5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.066154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.066642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.066922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.067193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.068220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:03.069469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.070958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.099211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 451/892] 10.222.0.128 () {56 vars in 1154 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:03.104919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.112044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.112820 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.113221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.113571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.114654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:03.115030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.116568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.128033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 442/893] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:03.129111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.236931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147d80c-a3a1-40d0-8fdd-a388c30bc1a6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.249827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c57ce5-1365-4505-bdd8-b1e36d2f5298 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.253144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:03.277509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:03.285610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.286555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.287036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.287668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.288667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.289380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.289738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.290092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.290976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:03.291360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.293189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.303794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 443/894] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:17:03 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:03.305091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.377153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.378585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.430129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488a09-be78-46aa-89fb-74f4d854e0a2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.476645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.477506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.477868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.478214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.479180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] RBAC: Adding query filter params (interface=public, service_id=abf2dc1c9fa74644bf2fd02db347d817, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:03.479591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.481093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.492572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 444/895] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:17:03 2022] GET /identity/v3/endpoints?service_id=abf2dc1c9fa74644bf2fd02db347d817&interface=public®ion_id=RegionOne => generated 195 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:03.493692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.501884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 452/896] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:03 2022] GET /identity/v3/auth/tokens => generated 1364 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 15:17:03.503184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.515586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:03.564897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:03.608314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95145d-b4fa-4cc4-bfa4-649cdfb95e72 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.610256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d1835-25d1-48fc-bb33-7d3de07bbcde None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.646658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.646848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.647285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.647490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.647585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.647764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.647926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.648337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.648436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.649999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.659789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.661416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.662489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 445/897] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:17:03 2022] GET /identity/v3/services => generated 1274 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:03.679555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:03.680582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:03.682039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:03.682039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:03.682039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:03.682039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:03.682039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 453/898] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:03.687359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.809873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb1c4c-817c-4fdd-90f3-991b6b4f30d8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:03.830412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:03.849541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:03.850177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:03.850469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:03.850750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:03.851457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:03.851771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:03.853173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:03.867557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 446/899] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:03.873507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:03.949288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:03.984497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e499752-b6f5-4fc4-9d6e-c20f758bdfe6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:04.020542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:04.021204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:04.021488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:04.021762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:04.029444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:04.031162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:17:04.043127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:04.044099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:04.045494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 454/900] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:04.050653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:04.163135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad8b12-92c6-44a7-9806-3dc8a9961a63 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:04.212581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2403b01-8ebb-4e02-a546-8529d6fb9e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:04.212581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403b01-8ebb-4e02-a546-8529d6fb9e20 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:04.212581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403b01-8ebb-4e02-a546-8529d6fb9e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:04.212581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403b01-8ebb-4e02-a546-8529d6fb9e20 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:04.212994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403b01-8ebb-4e02-a546-8529d6fb9e20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:04.212994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403b01-8ebb-4e02-a546-8529d6fb9e20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:04.212994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403b01-8ebb-4e02-a546-8529d6fb9e20 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:04.224829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 447/901] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:17:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:04.230383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:04.340077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26c9ce-1dba-4af2-ae5b-12940d1682cb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:04.368840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:04.376665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568a186-6a9d-417b-b6a3-1858b8a0f105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:04.377340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a186-6a9d-417b-b6a3-1858b8a0f105 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:04.377628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a186-6a9d-417b-b6a3-1858b8a0f105 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:04.377902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a186-6a9d-417b-b6a3-1858b8a0f105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:04.378697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568a186-6a9d-417b-b6a3-1858b8a0f105 None admin] RBAC: Adding query filter params (role.id=e08dc961ff534e868714c89db2909627, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=f35a6f8e44fa42dca9ab2ea540ba57ac) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:04.379011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568a186-6a9d-417b-b6a3-1858b8a0f105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:04.380625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568a186-6a9d-417b-b6a3-1858b8a0f105 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:04.386819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 455/902] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:04 2022] GET /identity/v3/role_assignments?user.id=f35a6f8e44fa42dca9ab2ea540ba57ac&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=e08dc961ff534e868714c89db2909627 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:04.446097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4a74a-9323-40ff-b6ee-3b3a08e9bc1a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:04.446466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4a74a-9323-40ff-b6ee-3b3a08e9bc1a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:04.446674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4a74a-9323-40ff-b6ee-3b3a08e9bc1a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:04.448549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 448/903] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:04.457748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950a8c-68ac-4de6-a953-a867eb289e93 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:04.458113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950a8c-68ac-4de6-a953-a867eb289e93 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:04.458604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950a8c-68ac-4de6-a953-a867eb289e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:04.478736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-39950a8c-68ac-4de6-a953-a867eb289e93 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:04.479911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39950a8c-68ac-4de6-a953-a867eb289e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:04.562855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 456/904] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:04 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:04.585517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4a74a-9323-40ff-b6ee-3b3a08e9bc1a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:04.631984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:04.697880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4a74a-9323-40ff-b6ee-3b3a08e9bc1a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:04.734703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:04.735282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:04.735572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:04.735855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:04.736535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:04.737913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:04.747919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:04.748944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:04.749794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 449/905] 10.222.0.128 () {56 vars in 1137 bytes} [Tue Jul 26 15:17:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:04.755348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39950a8c-68ac-4de6-a953-a867eb289e93 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:04.868205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39950a8c-68ac-4de6-a953-a867eb289e93 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:04.888877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:04.910121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:04.910773 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:04.911158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:04.911441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:04.912220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:04.912582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:04.914001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:04.925265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 457/906] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:04 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:04.931088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:05.042304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c419d7-f78d-474b-b33f-093d5c692f28 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:05.052840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:05.080453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ee58-97c5-4349-beb7-a38c37f3a425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:05.081149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ee58-97c5-4349-beb7-a38c37f3a425 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.081436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ee58-97c5-4349-beb7-a38c37f3a425 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.081711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ee58-97c5-4349-beb7-a38c37f3a425 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.082404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399ee58-97c5-4349-beb7-a38c37f3a425 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:05.082711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399ee58-97c5-4349-beb7-a38c37f3a425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:05.084139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399ee58-97c5-4349-beb7-a38c37f3a425 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:05.096355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:05.096819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 450/907] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:04 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:05.125146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd3003-5e3a-496f-b608-76c215fd9b0a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.125484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd3003-5e3a-496f-b608-76c215fd9b0a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.125708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd3003-5e3a-496f-b608-76c215fd9b0a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.127318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 451/908] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:05.132276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642fba-bd82-4c4f-960e-80632629e927 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.132603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642fba-bd82-4c4f-960e-80632629e927 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.132884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642fba-bd82-4c4f-960e-80632629e927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.151842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-22642fba-bd82-4c4f-960e-80632629e927 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:05.153184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-22642fba-bd82-4c4f-960e-80632629e927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:05.221223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cda7b2-7256-464a-9cde-ca645ab48262 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:05.237815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 452/909] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:05 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:05.262080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642fba-bd82-4c4f-960e-80632629e927 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:05.263723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:05.264362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.264852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.265128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.266898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] RBAC: Adding query filter params (interface=public, service_id=f98975962b214988a3453b7b39089c2e, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:05.267220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:05.268644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:05.275545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 458/910] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:17:05 2022] GET /identity/v3/endpoints?service_id=f98975962b214988a3453b7b39089c2e&interface=public®ion_id=RegionOne => generated 195 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:05.283801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:05.352925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:05.395804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb48df-0401-4e1b-bafe-b2c5b43a60b3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:05.397768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642fba-bd82-4c4f-960e-80632629e927 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:05.419036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:05.434231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ead34-d626-45b7-9b45-653c5179ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:05.434446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25682b2-aae9-4b6f-9b1b-75aec45cf975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:05.434859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ead34-d626-45b7-9b45-653c5179ab20 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.435141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ead34-d626-45b7-9b45-653c5179ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.435245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25682b2-aae9-4b6f-9b1b-75aec45cf975 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.436329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ead34-d626-45b7-9b45-653c5179ab20 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.436426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25682b2-aae9-4b6f-9b1b-75aec45cf975 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.436426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25682b2-aae9-4b6f-9b1b-75aec45cf975 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.436600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ead34-d626-45b7-9b45-653c5179ab20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:05.436681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25682b2-aae9-4b6f-9b1b-75aec45cf975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:05.437500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ead34-d626-45b7-9b45-653c5179ab20 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:05.437940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25682b2-aae9-4b6f-9b1b-75aec45cf975 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:05.444326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 459/911] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:17:05 2022] GET /identity/v3/services => generated 1274 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:05.523431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 453/912] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:17:05 2022] POST /identity/v3/services => generated 242 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:05.834192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d3436-7259-4cf8-a6f1-7b0e308b8aab None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.834713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d3436-7259-4cf8-a6f1-7b0e308b8aab None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.834819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d3436-7259-4cf8-a6f1-7b0e308b8aab None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.836360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 460/913] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:05.841664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d51068-121a-4122-94aa-b6fa293bf6c9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:05.842255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d51068-121a-4122-94aa-b6fa293bf6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:05.842255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d51068-121a-4122-94aa-b6fa293bf6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:05.861314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-54d51068-121a-4122-94aa-b6fa293bf6c9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:05.862513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-54d51068-121a-4122-94aa-b6fa293bf6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:05.946715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 454/914] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:05 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:05.970944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d3436-7259-4cf8-a6f1-7b0e308b8aab None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:06.085725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d3436-7259-4cf8-a6f1-7b0e308b8aab None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:06.096699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:06.125255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:06.126056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:06.126339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:06.126617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:06.127254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:06.128707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:06.140650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:06.142900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:06.144698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 461/915] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:17:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:06.148074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d51068-121a-4122-94aa-b6fa293bf6c9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:06.239732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:06.259101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d51068-121a-4122-94aa-b6fa293bf6c9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:06.297457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:06.298254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:06.298536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:06.298962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:06.299731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:06.300039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:06.301449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:06.306952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 455/916] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:17:06 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:06.312771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:06.422960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572a64d-ddd1-4a2a-9fde-46de86716b41 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:06.468142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2066839-f170-47cd-b3b9-a90d92effda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:06.468802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066839-f170-47cd-b3b9-a90d92effda0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:06.469082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066839-f170-47cd-b3b9-a90d92effda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:06.469359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066839-f170-47cd-b3b9-a90d92effda0 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:06.470047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2066839-f170-47cd-b3b9-a90d92effda0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:06.470504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2066839-f170-47cd-b3b9-a90d92effda0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:06.471884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2066839-f170-47cd-b3b9-a90d92effda0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:06.477548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 462/917] 10.222.0.128 () {56 vars in 1119 bytes} [Tue Jul 26 15:17:06 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:06.483345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:06.529255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ef2e-b846-4376-9d51-fef5596afcd4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:06.529598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ef2e-b846-4376-9d51-fef5596afcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:06.529833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ef2e-b846-4376-9d51-fef5596afcd4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:06.531286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 463/918] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:06 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:06.536542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266897e7-2935-4f61-b2fb-b86723e0fbd1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:06.536958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266897e7-2935-4f61-b2fb-b86723e0fbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:06.537343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266897e7-2935-4f61-b2fb-b86723e0fbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:06.557900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-266897e7-2935-4f61-b2fb-b86723e0fbd1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:06.559676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-266897e7-2935-4f61-b2fb-b86723e0fbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:06.593506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2065e-a553-4c00-ac8a-dd7724ae4416 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:06.595279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:06.638790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:06.639415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:06.639697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:06.639968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:06.640731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:06.640939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 464/919] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:06 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:06.642159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:06.665119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266897e7-2935-4f61-b2fb-b86723e0fbd1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:06.740499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 456/920] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:17:06 2022] POST /identity/v3/endpoints => generated 348 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:06.789170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266897e7-2935-4f61-b2fb-b86723e0fbd1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:06.841794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:06.843451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:06.843967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:06.844500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:06.858148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:06.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:06.864860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:06.865726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:06.872703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 465/921] 10.222.0.128 () {56 vars in 1125 bytes} [Tue Jul 26 15:17:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:06.877478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:06.889515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:06.993598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeb644-c3ed-47ee-8c8e-b3149fa650f1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:07.030914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:07.031546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.031826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.032101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.032868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:07.033220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:07.034691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:07.044027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 457/922] 10.222.0.128 () {56 vars in 1120 bytes} [Tue Jul 26 15:17:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:07.049757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:07.081811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:07.162380 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a63ef6-e57e-48fb-88fb-5bed905a99dc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:07.200812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:07.201663 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.201954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.202237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.213217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:07.214696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:07.233563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:07.234635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:07.235559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:07.235559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:07.235559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:07.235559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:07.237880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 466/923] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:07.246520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:07.314217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:07.355975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13671-6538-4bf1-b617-bd9ec188f3d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:07.393401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:07.394380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.394846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.395304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.396207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:07.396735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:07.398405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:07.416567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 458/924] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:07.418403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:07.514496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:07.527040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ce5e-756e-43e6-a672-a6990bdba5ff None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:07.563236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e53c1-2cd1-4aed-aea8-aabd393ee2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:07.563897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e53c1-2cd1-4aed-aea8-aabd393ee2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.564180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e53c1-2cd1-4aed-aea8-aabd393ee2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.564477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e53c1-2cd1-4aed-aea8-aabd393ee2e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.572654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e53c1-2cd1-4aed-aea8-aabd393ee2e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:07.574306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e53c1-2cd1-4aed-aea8-aabd393ee2e9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-812e53c1-2cd1-4aed-aea8-aabd393ee2e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:17:07.587030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:07.588027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:07.588999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 467/925] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:17:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:07.594645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:07.672889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813055b4-9757-4327-82c1-aab75863002f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.673230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813055b4-9757-4327-82c1-aab75863002f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.673450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813055b4-9757-4327-82c1-aab75863002f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.675020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 468/926] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:07.681626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbf5c7-c20d-4a36-9a6f-506ba7a5d87f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.681902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbf5c7-c20d-4a36-9a6f-506ba7a5d87f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.682177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbf5c7-c20d-4a36-9a6f-506ba7a5d87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.702443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a1bbf5c7-c20d-4a36-9a6f-506ba7a5d87f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:07.703628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a1bbf5c7-c20d-4a36-9a6f-506ba7a5d87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:07.714728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab36008-0246-478b-bdaa-a264f6d30b38 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:07.748084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:07.763112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:07.764566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.765336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.765960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.767355 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:07.768198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:07.770344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:07.776021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 469/927] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:07 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:07.776021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff25a5-f011-4a0e-856e-8c5e581d848c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.776021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff25a5-f011-4a0e-856e-8c5e581d848c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.776021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff25a5-f011-4a0e-856e-8c5e581d848c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.781922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 470/928] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:07 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:07.787719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94574ed0-a1a0-43f1-8d8e-f91f39b60a6c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.788010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94574ed0-a1a0-43f1-8d8e-f91f39b60a6c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.788284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94574ed0-a1a0-43f1-8d8e-f91f39b60a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.800063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 459/929] 10.222.0.128 () {56 vars in 1154 bytes} [Tue Jul 26 15:17:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:07.801624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:07.811454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-94574ed0-a1a0-43f1-8d8e-f91f39b60a6c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:07.813034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-94574ed0-a1a0-43f1-8d8e-f91f39b60a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:07.879179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:07.892519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 471/930] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:07 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:07.892519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94574ed0-a1a0-43f1-8d8e-f91f39b60a6c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:07.917929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a711e38-bcb6-445d-9771-0f4a55290579 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:07.954544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:07.955192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:07.955474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:07.955745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:07.956372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:07.957759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:07.967799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:07.968851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:07.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 460/931] 10.222.0.128 () {56 vars in 1137 bytes} [Tue Jul 26 15:17:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:07.970801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.023600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94574ed0-a1a0-43f1-8d8e-f91f39b60a6c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.027759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:08.063753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.064400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.064719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.064996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.075543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.077101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.090669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6e96-24c8-4c0c-8864-a3cbf365869a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:08.092095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:08.093167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:08.094045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:08.094045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:08.094045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:08.094045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:08.094045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 472/932] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:08.094045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.134630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.135523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.135977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.136406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.137076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.139029 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.147955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:08.148504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:08.149468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:08.150301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 461/933] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:08.151103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.211636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aec1ce-8324-427e-bdaa-559da2c41d3a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.250041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.250675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.250961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.251234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.251917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:08.252226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.253670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.263608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 473/934] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:08.263608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.279802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296564b-e770-4201-a2ca-ca05587b08d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.295083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:08.330602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.331263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.331545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.331993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.332715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:08.332860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.335034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.355624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 462/935] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:08.355624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.382013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb65471-e097-406c-a785-be56be8630a3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.393436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:08.419929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.420692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.420996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.421259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.422031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:08.422339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.423716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.433651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 474/936] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:08.433651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.476939 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef43f49-74d8-4a8c-813a-935e312bd173 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.517540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.518526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.519009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.519608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.520380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:08.520800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.522338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.533417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 463/937] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:08.534564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.546645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c73c0-a78c-4adc-b887-e483170b9dfc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.572588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:08.582512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.583168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.583456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.583735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.590616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.592810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:17:08.604786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:08.605742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 475/938] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:08.607943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.655566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:08.657889 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038e4ec-53fa-4ccc-861f-bfe2ffddbd81 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.692125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.692789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.693085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.693364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.694038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:08.694352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.695719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.701695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 464/939] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:08.702486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.735932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c91635-b203-4dcd-91c7-83959fbc8fdb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.774798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.775666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.776102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.776699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.777748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.779804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.810587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:08.810900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff3f5e-fad4-4fa6-925d-87cb29a93ff1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:08.846063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:08.853054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:08.853054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:08.853054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:08.853054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:08.853054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:08.853054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:08.861152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 465/940] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:08.861958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.918646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 476/941] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:17:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:08.920302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:08.987273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e7b0f-0669-4b29-8dcf-ecfe90f7f579 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:09.035575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e022a6-266b-4202-81cf-d3f2bd318942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:09.036215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e022a6-266b-4202-81cf-d3f2bd318942 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.036544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e022a6-266b-4202-81cf-d3f2bd318942 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.036822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e022a6-266b-4202-81cf-d3f2bd318942 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.037602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e022a6-266b-4202-81cf-d3f2bd318942 None admin] RBAC: Adding query filter params (interface=public, service_id=81d575f5483d46fd8fcfb799b66964b1, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:09.037978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e022a6-266b-4202-81cf-d3f2bd318942 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:09.039399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e022a6-266b-4202-81cf-d3f2bd318942 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:09.050414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 466/942] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:17:08 2022] GET /identity/v3/endpoints?service_id=81d575f5483d46fd8fcfb799b66964b1&interface=public®ion_id=RegionOne => generated 195 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:09.050414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971824-92a4-4958-b3ee-57e55cdac971 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.050414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971824-92a4-4958-b3ee-57e55cdac971 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.050414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971824-92a4-4958-b3ee-57e55cdac971 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.050414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 467/943] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:09.051454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71971824-92a4-4958-b3ee-57e55cdac971 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:09.056151 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:09.070710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22c05-fdba-4502-9428-6bf64610c5c7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:09.077189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:09.112441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:09.114485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.114485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.114485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/f35a6f8e44fa42dca9ab2ea540ba57ac/roles/e08dc961ff534e868714c89db2909627` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.117221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=f35a6f8e44fa42dca9ab2ea540ba57ac, role_id=e08dc961ff534e868714c89db2909627)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:09.118780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:09.171172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71971824-92a4-4958-b3ee-57e55cdac971 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:09.206595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1be14f-9da7-42b3-a687-d85881559648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:09.207344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1be14f-9da7-42b3-a687-d85881559648 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.207629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1be14f-9da7-42b3-a687-d85881559648 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.207904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1be14f-9da7-42b3-a687-d85881559648 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.208592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1be14f-9da7-42b3-a687-d85881559648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:09.209953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1be14f-9da7-42b3-a687-d85881559648 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:09.216370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 468/944] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:17:09 2022] GET /identity/v3/services => generated 1504 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:09.217720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e80f-2433-407f-a147-fe986c9ba7bb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.218065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e80f-2433-407f-a147-fe986c9ba7bb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.218427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e80f-2433-407f-a147-fe986c9ba7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.237793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 477/945] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:17:08 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/f35a6f8e44fa42dca9ab2ea540ba57ac/roles/e08dc961ff534e868714c89db2909627 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:09.251744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a9d4e80f-2433-407f-a147-fe986c9ba7bb None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:09.257796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a9d4e80f-2433-407f-a147-fe986c9ba7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:09.373665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 469/946] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:09 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:09.397943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:09.531476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618c47-80bd-49d6-a9ab-cd8b71488860 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:09.569107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:09.569983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.570419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.570846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.571691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:09.573344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:09.576928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING keystone.server.flask.common [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 15:17:09.637373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.sql.core [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 15:17:09.637373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 15:17:09.637373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [parameters: {'id': '72e21cf319014d7b9dee93266fc078f5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 15:17:09.637373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80377) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 72e21cf319014d7b9dee93266fc078f5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 72e21cf319014d7b9dee93266fc078f5. Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 15:17:09.659128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 15:17:09.660183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application yield resource Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise self.value Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application session.commit() Jul 26 15:17:09.661495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.flush() Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application compat.raise_( Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 15:17:09.662652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 15:17:09.663790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 15:17:09.664698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [parameters: {'id': '72e21cf319014d7b9dee93266fc078f5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:09.665580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:09.666366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 72e21cf319014d7b9dee93266fc078f5. Jul 26 15:17:09.667307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:09.669876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 478/947] 10.222.0.128 () {60 vars in 1145 bytes} [Tue Jul 26 15:17:09 2022] POST /identity/v3/projects => generated 262 bytes in 272 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:09.675589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4e80f-2433-407f-a147-fe986c9ba7bb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:09.690977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:09.789605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4e80f-2433-407f-a147-fe986c9ba7bb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:09.826408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecbe49-ee9d-462e-b28b-3524825ec512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:09.827387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecbe49-ee9d-462e-b28b-3524825ec512 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.827846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecbe49-ee9d-462e-b28b-3524825ec512 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.828295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecbe49-ee9d-462e-b28b-3524825ec512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.837347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecbe49-ee9d-462e-b28b-3524825ec512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:09.839072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecbe49-ee9d-462e-b28b-3524825ec512 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-baecbe49-ee9d-462e-b28b-3524825ec512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:17:09.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:09.853932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:09.854862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:09.857598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 470/948] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:09.867302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:09.884936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:09.979096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c6ece-d750-47cf-a67c-6e29517255ca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:10.017504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc63c99-e702-4db1-b2ae-94f2efe8f93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:10.018407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63c99-e702-4db1-b2ae-94f2efe8f93c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:10.018834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63c99-e702-4db1-b2ae-94f2efe8f93c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:10.019255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63c99-e702-4db1-b2ae-94f2efe8f93c None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:10.020230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc63c99-e702-4db1-b2ae-94f2efe8f93c None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:10.020706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc63c99-e702-4db1-b2ae-94f2efe8f93c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:10.022498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc63c99-e702-4db1-b2ae-94f2efe8f93c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:10.045307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 479/949] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:17:09 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:11.052197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b8153-e4fc-460a-a25a-3a6c8282e3cc None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.052754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b8153-e4fc-460a-a25a-3a6c8282e3cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.052754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b8153-e4fc-460a-a25a-3a6c8282e3cc None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.054445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 471/950] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:11.060231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e90da-5924-4f9f-b472-2d602d37f443 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.060703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e90da-5924-4f9f-b472-2d602d37f443 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.060780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e90da-5924-4f9f-b472-2d602d37f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.080038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-475e90da-5924-4f9f-b472-2d602d37f443 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:11.081590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-475e90da-5924-4f9f-b472-2d602d37f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:11.149370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:11.157239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 480/951] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:11 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:11.179822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b8153-e4fc-460a-a25a-3a6c8282e3cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:11.260511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:11.294797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b8153-e4fc-460a-a25a-3a6c8282e3cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:11.332092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50440915-b4a3-42d3-9194-41942f987183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:11.332710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440915-b4a3-42d3-9194-41942f987183 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.333002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440915-b4a3-42d3-9194-41942f987183 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.333439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440915-b4a3-42d3-9194-41942f987183 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.334142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50440915-b4a3-42d3-9194-41942f987183 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:11.335533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50440915-b4a3-42d3-9194-41942f987183 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-50440915-b4a3-42d3-9194-41942f987183 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:11.345732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:11.346909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:11.348187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 472/952] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:11.353364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e90da-5924-4f9f-b472-2d602d37f443 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:11.462202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd69d7-d4d8-44d4-a913-e307f4b43e8d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.462579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd69d7-d4d8-44d4-a913-e307f4b43e8d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.463424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd69d7-d4d8-44d4-a913-e307f4b43e8d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.464804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 473/953] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:11.470721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b72cde-7c54-450b-988d-be09aef7cfb0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.471017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b72cde-7c54-450b-988d-be09aef7cfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.471302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b72cde-7c54-450b-988d-be09aef7cfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.478076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e90da-5924-4f9f-b472-2d602d37f443 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:11.491209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-68b72cde-7c54-450b-988d-be09aef7cfb0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:11.492498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-68b72cde-7c54-450b-988d-be09aef7cfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:11.517678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319ac59-0441-4e28-8d0d-74dfb5df69a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:11.518366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ac59-0441-4e28-8d0d-74dfb5df69a5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.518592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ac59-0441-4e28-8d0d-74dfb5df69a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.518875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ac59-0441-4e28-8d0d-74dfb5df69a5 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.519568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319ac59-0441-4e28-8d0d-74dfb5df69a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:11.519879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319ac59-0441-4e28-8d0d-74dfb5df69a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:11.521422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319ac59-0441-4e28-8d0d-74dfb5df69a5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:11.530220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 481/954] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:17:11 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:11.530220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab23a2-dba3-490d-8b8f-4d05ba23596f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.530220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab23a2-dba3-490d-8b8f-4d05ba23596f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.530220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab23a2-dba3-490d-8b8f-4d05ba23596f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.530220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 482/955] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:11.535553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf88ba8-fc77-4951-b089-ba888669fdb6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.535839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf88ba8-fc77-4951-b089-ba888669fdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.536126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf88ba8-fc77-4951-b089-ba888669fdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.554759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7bf88ba8-fc77-4951-b089-ba888669fdb6 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:11.556051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7bf88ba8-fc77-4951-b089-ba888669fdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:11.591440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 474/956] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:11 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:11.592210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b72cde-7c54-450b-988d-be09aef7cfb0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:11.621658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:11.644490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 483/957] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:11 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:11.644490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf88ba8-fc77-4951-b089-ba888669fdb6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:11.674031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:11.712913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b72cde-7c54-450b-988d-be09aef7cfb0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:11.754003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:11.754937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.755227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.755566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.756273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:11.756621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:11.758006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:11.763257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 475/958] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:17:11 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:11.764076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:11.765025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf88ba8-fc77-4951-b089-ba888669fdb6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:11.824049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:11.825790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.827080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.827698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.828922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:11.832237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:11.846892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:11.847963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:11.848846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:11.850335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 484/959] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:11.857533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:11.890803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114ab5-97c9-42bd-a6de-52f302b0af33 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:11.921944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:11.926999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:11.938670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:11.939366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:11.939638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:11.939918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:11.956831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:11.962746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:11.963744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:11.964802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 476/960] 10.222.0.128 () {56 vars in 1125 bytes} [Tue Jul 26 15:17:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:11.971417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:11.987241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40901a-873d-4453-a2f8-b282c4f637fa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.034143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.035504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.036251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.036777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.038100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:12.038646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.040839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.049704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 485/961] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:11 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:12.060494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.098876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2a3a6-dc08-439e-9c83-5fef5cd9a57d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.137868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.138592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.138878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.139154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.139885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:12.140197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.141736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.156867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 477/962] 10.222.0.128 () {56 vars in 1120 bytes} [Tue Jul 26 15:17:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:12.158331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.174156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344e50-c1b7-41cc-976b-37e816a6b018 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.192599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:12.212837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceb411-cac6-4655-bbc7-54bac15e2737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.213467 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceb411-cac6-4655-bbc7-54bac15e2737 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.213753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceb411-cac6-4655-bbc7-54bac15e2737 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.214032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceb411-cac6-4655-bbc7-54bac15e2737 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.214720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceb411-cac6-4655-bbc7-54bac15e2737 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:12.215028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceb411-cac6-4655-bbc7-54bac15e2737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.217062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceb411-cac6-4655-bbc7-54bac15e2737 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.224998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 486/963] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:12 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:12.224998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66fe74-a729-45a5-8297-98ed17473700 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.224998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66fe74-a729-45a5-8297-98ed17473700 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.224998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66fe74-a729-45a5-8297-98ed17473700 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.229730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:12.229852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 487/964] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:12.229852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66fe74-a729-45a5-8297-98ed17473700 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.285180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b2d91-403d-4b32-918c-008d615bc589 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.329578 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7f700-b1a9-43b4-a099-9b3ccf8ad2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.329852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7f700-b1a9-43b4-a099-9b3ccf8ad2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.329988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7f700-b1a9-43b4-a099-9b3ccf8ad2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.330345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7f700-b1a9-43b4-a099-9b3ccf8ad2d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.344899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7f700-b1a9-43b4-a099-9b3ccf8ad2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.346370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7f700-b1a9-43b4-a099-9b3ccf8ad2d0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8fa7f700-b1a9-43b4-a099-9b3ccf8ad2d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:12.362687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:12.364155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:12.365568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:12.365568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:12.365568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:12.365568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:12.365568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 478/965] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:12.365858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60966192-bc78-4fc4-a9a1-324f67eaba1a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.366148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60966192-bc78-4fc4-a9a1-324f67eaba1a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.366312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60966192-bc78-4fc4-a9a1-324f67eaba1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.379454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66fe74-a729-45a5-8297-98ed17473700 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.392118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-60966192-bc78-4fc4-a9a1-324f67eaba1a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:12.393368 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-60966192-bc78-4fc4-a9a1-324f67eaba1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:12.419015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.420394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.420704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.420982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.421716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.423352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.471155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 479/966] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:12 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:12.472012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60966192-bc78-4fc4-a9a1-324f67eaba1a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.510456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:12.512265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 488/967] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:17:12 2022] POST /identity/v3/endpoints => generated 351 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:12.513231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.574404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:12.613867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60966192-bc78-4fc4-a9a1-324f67eaba1a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.653637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4078946-ba42-4f1e-81d1-5f0cf8583c08 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.658976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.659920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.660364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.660778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.661582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:12.662015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.663572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.683366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 480/968] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:12.690791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.699064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.699642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.700809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.701090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.701786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.703250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.745469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.sql.core [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 15:17:12.745469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 15:17:12.745469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [parameters: {'user_id': '024d9e1841af45498527479706fd6adb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 15:17:12.745469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80377) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Jul 26 15:17:12.753314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 15:17:12.754399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application yield resource Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise self.value Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application session.commit() Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 15:17:12.755390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.flush() Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application compat.raise_( Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 15:17:12.756293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 15:17:12.757252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 15:17:12.759820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [parameters: {'user_id': '024d9e1841af45498527479706fd6adb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:12.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 15:17:12.765149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 489/969] 10.222.0.128 () {60 vars in 1136 bytes} [Tue Jul 26 15:17:12 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:12.768963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.824222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:12.836597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04ab55-5e4c-4e0f-a0c0-02db678a86e6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.880805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.881670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.882449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.882981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.884252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:12.892011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479a7a9-f47f-491a-95f6-cab38351e81f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:12.896252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.897754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:17:12.914291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:12.915311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:12.916343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 481/970] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:17:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:12.921818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:12.947216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:12.948579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:12.949402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:12.950083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:12.964879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:12.969180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:17:12.990322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:12.991474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 15:17:12.992487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:12.998305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 490/971] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:17:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:13.004459 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:13.062471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07a595-c488-48f7-9224-7a5821911fb9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:13.117786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:13.119399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:13.120299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:13.120748 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:13.123468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:13.123977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:13.126139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:13.143905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cea30-65c2-4cbe-ab70-3284dc9c6358 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:13.154526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 482/972] 10.222.0.128 () {56 vars in 1154 bytes} [Tue Jul 26 15:17:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:13.160906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:13.178485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:13.194033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:13.195107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:13.195596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:13.196082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:13.197226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:13.197765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:13.199590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:13.250224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 491/973] 10.222.0.128 () {56 vars in 1106 bytes} [Tue Jul 26 15:17:13 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:13.320428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b3e3-2f3c-45cf-a502-18f2a0646035 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:13.376528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:13.377503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:13.378001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:13.378517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:13.493242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:13.494886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:13.513275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:13.614724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 483/974] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:13 2022] GET /identity/v3/auth/tokens => generated 1466 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 15:17:13.620878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:13.639263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:13.740822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87576acf-bc55-42ad-ac6b-9ca3ba20062b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:13.779400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80a1e9-2432-4e16-b378-805b13be0067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:13.780059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a1e9-2432-4e16-b378-805b13be0067 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:13.782484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a1e9-2432-4e16-b378-805b13be0067 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:13.782484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a1e9-2432-4e16-b378-805b13be0067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:13.798019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80a1e9-2432-4e16-b378-805b13be0067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:13.799597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80a1e9-2432-4e16-b378-805b13be0067 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1f80a1e9-2432-4e16-b378-805b13be0067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:13.818698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:13.819723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:13.821725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:13.821725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:13.821725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:13.821725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:13.824756 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 492/975] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:13.825959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:13.936642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd314-960a-424f-b349-b5db35fd3806 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:13.944953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c826ccb-8d7a-490f-a52d-e4bfcb34e1ae None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:13.945285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c826ccb-8d7a-490f-a52d-e4bfcb34e1ae None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:13.945521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c826ccb-8d7a-490f-a52d-e4bfcb34e1ae None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:13.947083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 493/976] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:13.952604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:13.952935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7f51-6626-4b13-897a-4ac0f4092716 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:13.953215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7f51-6626-4b13-897a-4ac0f4092716 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:13.953969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7f51-6626-4b13-897a-4ac0f4092716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:13.972445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d9ad7f51-6626-4b13-897a-4ac0f4092716 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:13.974007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d9ad7f51-6626-4b13-897a-4ac0f4092716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:13.977293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:13.977928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:13.978218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:13.978498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:13.979202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:13.979516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:13.980964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:13.996776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 484/977] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:14.002372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:14.043517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:14.056682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 494/978] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:13 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:14.082177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7f51-6626-4b13-897a-4ac0f4092716 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:14.120763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0a1d0-db25-401f-98f9-c2d477dfa44e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:14.160688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:14.161344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:14.161634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:14.161918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:14.169065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:14.170715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:17:14.184196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:14.185531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:14.186493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 485/979] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:14.192118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:14.196339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7f51-6626-4b13-897a-4ac0f4092716 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:14.233080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:14.242874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:14.243499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:14.243777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:14.244052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:14.244789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:14.246159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:14.306801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76635205-5a7f-40a9-b18e-590aa1499b47 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:14.335329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 495/980] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:17:14 2022] POST /identity/v3/services => generated 243 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:14.357162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7e594-df11-4477-b8da-27700755b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:14.357812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e594-df11-4477-b8da-27700755b0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:14.358096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e594-df11-4477-b8da-27700755b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:14.358465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e594-df11-4477-b8da-27700755b0e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:14.359413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7e594-df11-4477-b8da-27700755b0e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:14.359853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7e594-df11-4477-b8da-27700755b0e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:14.361701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7e594-df11-4477-b8da-27700755b0e5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:14.382551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 486/981] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:14.384396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:14.459767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:14.513767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a35656-4c22-44b0-8550-8877eebcdfe3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:14.550746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab7286-d5b7-4804-aa25-7abad1bca255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:14.551385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab7286-d5b7-4804-aa25-7abad1bca255 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:14.551675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab7286-d5b7-4804-aa25-7abad1bca255 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:14.551958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab7286-d5b7-4804-aa25-7abad1bca255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:14.552864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab7286-d5b7-4804-aa25-7abad1bca255 None admin] RBAC: Adding query filter params (role.id=e08dc961ff534e868714c89db2909627, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=f35a6f8e44fa42dca9ab2ea540ba57ac) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:14.553185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab7286-d5b7-4804-aa25-7abad1bca255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:14.554629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab7286-d5b7-4804-aa25-7abad1bca255 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:14.561455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 496/982] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:14 2022] GET /identity/v3/role_assignments?user.id=f35a6f8e44fa42dca9ab2ea540ba57ac&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=e08dc961ff534e868714c89db2909627 => generated 610 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:14.667184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c74e3-ffa2-4e80-87c6-1a017a33486f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:14.667475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c74e3-ffa2-4e80-87c6-1a017a33486f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:14.667762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c74e3-ffa2-4e80-87c6-1a017a33486f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:14.670263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 487/983] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:14.676226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db222b0-081e-4973-82a0-2b3e84ae3a88 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:14.676545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db222b0-081e-4973-82a0-2b3e84ae3a88 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:14.676844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db222b0-081e-4973-82a0-2b3e84ae3a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:14.698122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2db222b0-081e-4973-82a0-2b3e84ae3a88 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:14.699371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2db222b0-081e-4973-82a0-2b3e84ae3a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:14.778716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 497/984] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:14 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:14.800930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c74e3-ffa2-4e80-87c6-1a017a33486f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:14.901157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:14.916223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c74e3-ffa2-4e80-87c6-1a017a33486f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:14.956681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:14.957338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:14.957624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:14.957904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:14.958540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:14.959908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:14.969230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:14.970281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:14.971117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 488/985] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:14.976573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db222b0-081e-4973-82a0-2b3e84ae3a88 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:15.030804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:15.088137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db222b0-081e-4973-82a0-2b3e84ae3a88 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:15.125089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:15.125718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:15.126004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:15.126283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:15.126976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:15.127291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:15.128825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:15.134128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 498/986] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:14 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:15.139918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:15.250099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55944fd0-5dbb-475d-b6de-c05e725ecd01 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:15.285635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:15.286263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:15.286552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:15.286831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:15.287524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:15.287835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:15.289232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:15.299399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 489/987] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:15 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:15.301542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:15.408405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fa472-621b-40d3-9785-8da59d5c2c1e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:15.424587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:15.445943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b3503-b596-433e-8806-4509ad197caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:15.446578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3503-b596-433e-8806-4509ad197caa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:15.446867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3503-b596-433e-8806-4509ad197caa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:15.447148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3503-b596-433e-8806-4509ad197caa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:15.447923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b3503-b596-433e-8806-4509ad197caa None admin] RBAC: Adding query filter params (interface=admin, service_id=81d575f5483d46fd8fcfb799b66964b1, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:15.448241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b3503-b596-433e-8806-4509ad197caa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:15.452521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b3503-b596-433e-8806-4509ad197caa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:15.462447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 499/988] 10.222.0.128 () {56 vars in 1254 bytes} [Tue Jul 26 15:17:15 2022] GET /identity/v3/endpoints?service_id=81d575f5483d46fd8fcfb799b66964b1&interface=admin®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:15.463784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:15.484303 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:15.572813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c90e3-58bd-417e-9266-9f6e476ca9d4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:15.609763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1cd4a-b6b9-4107-b16c-e277a0d7650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:15.610398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1cd4a-b6b9-4107-b16c-e277a0d7650a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:15.610690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1cd4a-b6b9-4107-b16c-e277a0d7650a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:15.610964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1cd4a-b6b9-4107-b16c-e277a0d7650a None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:15.611628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1cd4a-b6b9-4107-b16c-e277a0d7650a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:15.613045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1cd4a-b6b9-4107-b16c-e277a0d7650a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:15.628506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 490/989] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:17:15 2022] GET /identity/v3/services => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:16.420612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f292ce7-3370-4b85-abd9-11642327f023 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:16.421137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f292ce7-3370-4b85-abd9-11642327f023 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:16.421216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f292ce7-3370-4b85-abd9-11642327f023 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:16.422538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 500/990] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:16.427880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6b08-82c6-4989-913d-c39af04c02b0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:16.428306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6b08-82c6-4989-913d-c39af04c02b0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:16.428553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6b08-82c6-4989-913d-c39af04c02b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:16.447525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c49a6b08-82c6-4989-913d-c39af04c02b0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:16.448749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c49a6b08-82c6-4989-913d-c39af04c02b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:16.523409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 491/991] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:16 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:16.546281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f292ce7-3370-4b85-abd9-11642327f023 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:16.650489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f292ce7-3370-4b85-abd9-11642327f023 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:16.692953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:16.693517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:16.693796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:16.694068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] PATH_INFO: `/v3/services/compute` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:16.694693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:16.696048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:16.705410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38582-ba0e-4ca2-a603-35ccf1d3bb26 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:16.705699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38582-ba0e-4ca2-a603-35ccf1d3bb26 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:16.705968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38582-ba0e-4ca2-a603-35ccf1d3bb26 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:16.706416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:16.707387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:16.708243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:16.709029 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 492/992] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:16.712436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 501/993] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:16.712791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f7d60-53ca-41a6-893e-fa7740cd46ff None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:16.713078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f7d60-53ca-41a6-893e-fa7740cd46ff None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:16.713174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:16.713357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f7d60-53ca-41a6-893e-fa7740cd46ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:16.733115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ce7f7d60-53ca-41a6-893e-fa7740cd46ff None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:16.734373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ce7f7d60-53ca-41a6-893e-fa7740cd46ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:16.737436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:16.759397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:16.813692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 493/994] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:16 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:16.829285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4322e6-7cfe-4ff2-b2ff-45c4b5a924db None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:16.836100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f7d60-53ca-41a6-893e-fa7740cd46ff None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:16.870016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:16.870949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:16.871744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:16.873034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:16.873289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:16.873612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:16.875011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:16.884545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 502/995] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:17:16 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:16.886708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:16.946417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f7d60-53ca-41a6-893e-fa7740cd46ff None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:16.987380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:16.988059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:16.988646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:16.988991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] PATH_INFO: `/v3/services/image` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:16.989560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:16.990965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:17.001459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:17.002513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:17.003358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 494/996] 10.222.0.128 () {56 vars in 1110 bytes} [Tue Jul 26 15:17:16 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:17.008918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:17.013365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:17.013500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d9242-4fc4-4c55-9593-838465d575c0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:17.021368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:17.053777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:17.054477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:17.055118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:17.055405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:17.056417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:17.056977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:17.058745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:17.065417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 503/997] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:17:16 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:17.071214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:17.120332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824b01c-de6b-46b5-8f39-3733c0f39012 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:17.158832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:17.159457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:17.159745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:17.160051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:17.160913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:17.161099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:17.162440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:17.172057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 495/998] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:17 2022] GET /identity/v3/services?name=image => generated 123 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:17.174140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:17.186107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7482d-8f8b-4d86-9174-f2bb8ea5bc58 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:17.224251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:17.225054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:17.225364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:17.225622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:17.226436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] RBAC: Adding query filter params (interface=public, service_id=7e4554daf7b84337be00a1851c03f7e4, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:17.226756 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:17.228147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:17.234355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 504/999] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:17:17 2022] GET /identity/v3/endpoints?service_id=7e4554daf7b84337be00a1851c03f7e4&interface=public®ion_id=RegionOne => generated 195 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:17.240188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:17.258708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:17.278488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:17.296945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbfdd2-071e-4abd-b402-53a13ce6ae5e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:17.333542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19b70a-ba54-49c4-8731-331e6fb7643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:17.334469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19b70a-ba54-49c4-8731-331e6fb7643e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:17.334939 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19b70a-ba54-49c4-8731-331e6fb7643e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:17.335216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19b70a-ba54-49c4-8731-331e6fb7643e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:17.335900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19b70a-ba54-49c4-8731-331e6fb7643e None admin] RBAC: Adding query filter params (type=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:17.336207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19b70a-ba54-49c4-8731-331e6fb7643e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:17.337605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19b70a-ba54-49c4-8731-331e6fb7643e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:17.352493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 496/1000] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:17 2022] GET /identity/v3/services?type=image => generated 123 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:17.356931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428167-1c1b-4367-bd40-7b602da1c22d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:17.398644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e733f6-351b-499f-8448-622065f49ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:17.399303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e733f6-351b-499f-8448-622065f49ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:17.399590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e733f6-351b-499f-8448-622065f49ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:17.399870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e733f6-351b-499f-8448-622065f49ec7 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:17.400572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e733f6-351b-499f-8448-622065f49ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:17.401980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e733f6-351b-499f-8448-622065f49ec7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:17.409558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 505/1001] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:17:17 2022] GET /identity/v3/services => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:17.719641 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfef5c-5bcd-4a87-869c-817378407699 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:17.720160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfef5c-5bcd-4a87-869c-817378407699 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:17.720267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfef5c-5bcd-4a87-869c-817378407699 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:17.721594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 497/1002] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:17.727166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4014c82-3ed5-40c7-aa11-38ac995c0d52 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:17.728265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4014c82-3ed5-40c7-aa11-38ac995c0d52 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:17.728265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4014c82-3ed5-40c7-aa11-38ac995c0d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:17.748791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b4014c82-3ed5-40c7-aa11-38ac995c0d52 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:17.750226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b4014c82-3ed5-40c7-aa11-38ac995c0d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:17.826236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 506/1003] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:17 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:17.848838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfef5c-5bcd-4a87-869c-817378407699 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:17.961799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfef5c-5bcd-4a87-869c-817378407699 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:17.994006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:18.000888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:18.001619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:18.001906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:18.002186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:18.007742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:18.007742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:18.013742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:18.014741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:18.015696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 498/1004] 10.222.0.128 () {56 vars in 1131 bytes} [Tue Jul 26 15:17:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:18.023354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4014c82-3ed5-40c7-aa11-38ac995c0d52 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:18.054684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:18.130946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4014c82-3ed5-40c7-aa11-38ac995c0d52 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:18.167237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:18.167862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:18.168199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:18.168488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:18.169187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:18.169505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:18.170878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:18.176069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 507/1005] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:18 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:18.181735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:18.286119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de18b5-6219-40e2-ae45-cfc9c78fe3a8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:18.320238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12252f64-d8a4-4730-b396-2aaf43302b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:18.320888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252f64-d8a4-4730-b396-2aaf43302b4b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:18.321175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252f64-d8a4-4730-b396-2aaf43302b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:18.321452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252f64-d8a4-4730-b396-2aaf43302b4b None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:18.322134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12252f64-d8a4-4730-b396-2aaf43302b4b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:18.322443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12252f64-d8a4-4730-b396-2aaf43302b4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:18.323794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12252f64-d8a4-4730-b396-2aaf43302b4b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:18.333783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 499/1006] 10.222.0.128 () {56 vars in 1127 bytes} [Tue Jul 26 15:17:18 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:18.334969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:18.431056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:18.440765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31e766-c597-4d09-9daa-f18ef79a328b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:18.484304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91d3b8-8888-416a-bcb4-b807269a9409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:18.485087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91d3b8-8888-416a-bcb4-b807269a9409 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:18.485331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91d3b8-8888-416a-bcb4-b807269a9409 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:18.485571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91d3b8-8888-416a-bcb4-b807269a9409 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:18.486307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91d3b8-8888-416a-bcb4-b807269a9409 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:18.487742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91d3b8-8888-416a-bcb4-b807269a9409 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:18.564575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 508/1007] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:17:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:19.394557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c02bc-bbcd-449e-a48b-b306175f2a3c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:19.395082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c02bc-bbcd-449e-a48b-b306175f2a3c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:19.395082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c02bc-bbcd-449e-a48b-b306175f2a3c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:19.396533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 500/1008] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:19.401523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ee2b-95b2-42e5-8245-bb799df271ba None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:19.401908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ee2b-95b2-42e5-8245-bb799df271ba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:19.402078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ee2b-95b2-42e5-8245-bb799df271ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:19.421962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7088ee2b-95b2-42e5-8245-bb799df271ba None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:19.423157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7088ee2b-95b2-42e5-8245-bb799df271ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:19.508374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 509/1009] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:19 2022] POST /identity/v3/auth/tokens => generated 1710 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:19.519554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35b265-9bce-41a9-acb5-049e61c18d3d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:19.519846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35b265-9bce-41a9-acb5-049e61c18d3d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:19.520178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35b265-9bce-41a9-acb5-049e61c18d3d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:19.521575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 501/1010] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:19.526893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e475f0-6de4-438f-8f84-01f0c51cc2ba None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:19.527191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e475f0-6de4-438f-8f84-01f0c51cc2ba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:19.528199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e475f0-6de4-438f-8f84-01f0c51cc2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:19.531617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35b265-9bce-41a9-acb5-049e61c18d3d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:19.546848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-82e475f0-6de4-438f-8f84-01f0c51cc2ba None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:19.548053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-82e475f0-6de4-438f-8f84-01f0c51cc2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:19.600041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:19.627001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 510/1011] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:19 2022] POST /identity/v3/auth/tokens => generated 1710 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:19.646008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35b265-9bce-41a9-acb5-049e61c18d3d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:19.649704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e475f0-6de4-438f-8f84-01f0c51cc2ba None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:19.661239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:19.689821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:19.690839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:19.691363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:19.691859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:19.692855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:19.694290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:19.755958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e475f0-6de4-438f-8f84-01f0c51cc2ba None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:19.800495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 502/1012] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:17:19 2022] POST /identity/v3/services => generated 243 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:19.800732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:19.800732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:19.800732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:19.800732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:19.801326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:19.802493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:19.812700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:19.813867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 15:17:19.814711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:19.815462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 511/1013] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:19.822057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:19.937756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16254df-73ef-45ad-a7fa-0279cea69a76 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:19.962175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:19.979366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:19.980067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:19.980350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:19.980841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:19.981623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:19.981939 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:19.983347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:19.988645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 503/1014] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:17:19 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:19.995690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:20.069635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:20.105691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193f23a-4e3e-443f-9f17-6ea43c4cec0c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:20.140237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fcfe5-4fca-4703-9cf5-59b921dc1a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:20.140982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fcfe5-4fca-4703-9cf5-59b921dc1a9a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:20.141188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fcfe5-4fca-4703-9cf5-59b921dc1a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:20.141447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fcfe5-4fca-4703-9cf5-59b921dc1a9a None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:20.142136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fcfe5-4fca-4703-9cf5-59b921dc1a9a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:20.142443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fcfe5-4fca-4703-9cf5-59b921dc1a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:20.143827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fcfe5-4fca-4703-9cf5-59b921dc1a9a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:20.149306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 512/1015] 10.222.0.128 () {56 vars in 1117 bytes} [Tue Jul 26 15:17:19 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:20.155686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:20.265162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4520-25f7-4d8e-a707-2c9d9d6ce31e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:20.301846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53793922-7c13-4061-ab41-3bebacb53019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:20.302890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53793922-7c13-4061-ab41-3bebacb53019 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:20.303385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53793922-7c13-4061-ab41-3bebacb53019 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:20.303950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53793922-7c13-4061-ab41-3bebacb53019 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:20.304980 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53793922-7c13-4061-ab41-3bebacb53019 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:20.306567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53793922-7c13-4061-ab41-3bebacb53019 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:20.420512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 504/1016] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:17:20 2022] POST /identity/v3/endpoints => generated 336 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:20.654583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cca22e-6043-46b6-a8e1-77576fdaf3ca None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:20.655081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cca22e-6043-46b6-a8e1-77576fdaf3ca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:20.655081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cca22e-6043-46b6-a8e1-77576fdaf3ca None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:20.656401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 513/1017] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:20.661938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016ab0-8bfb-456e-857c-9646a5da0acc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:20.662226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016ab0-8bfb-456e-857c-9646a5da0acc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:20.662504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016ab0-8bfb-456e-857c-9646a5da0acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:20.683325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-99016ab0-8bfb-456e-857c-9646a5da0acc None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:20.684624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-99016ab0-8bfb-456e-857c-9646a5da0acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:20.722900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:20.764636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 505/1018] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:20 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:20.787131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cca22e-6043-46b6-a8e1-77576fdaf3ca None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:20.897590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cca22e-6043-46b6-a8e1-77576fdaf3ca None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:20.927823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:20.933337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890a40-110f-4708-a357-f20865e11a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:20.939254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890a40-110f-4708-a357-f20865e11a1d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:20.939254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890a40-110f-4708-a357-f20865e11a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:20.939254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890a40-110f-4708-a357-f20865e11a1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:20.939254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890a40-110f-4708-a357-f20865e11a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:20.939254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890a40-110f-4708-a357-f20865e11a1d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:20.939872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 514/1019] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:20 2022] GET /identity/v3/domains/default => generated 200 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:20.944925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99016ab0-8bfb-456e-857c-9646a5da0acc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:21.053781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99016ab0-8bfb-456e-857c-9646a5da0acc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:21.088355 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c0a4d-6d63-4f2d-90e6-1c67030f69d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:21.089058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0a4d-6d63-4f2d-90e6-1c67030f69d2 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:21.089349 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0a4d-6d63-4f2d-90e6-1c67030f69d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:21.090150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0a4d-6d63-4f2d-90e6-1c67030f69d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:21.090981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c0a4d-6d63-4f2d-90e6-1c67030f69d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:21.092416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c0a4d-6d63-4f2d-90e6-1c67030f69d2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:21.187038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 506/1020] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:17:20 2022] POST /identity/v3/projects => generated 312 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:21.907977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef591bc-af98-4ca8-9aee-14a2a0c96d31 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:21.908553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef591bc-af98-4ca8-9aee-14a2a0c96d31 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:21.908635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef591bc-af98-4ca8-9aee-14a2a0c96d31 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:21.910128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 515/1021] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:21.915533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b003a-77d5-4c6d-9c91-7e29630f1093 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:21.915962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b003a-77d5-4c6d-9c91-7e29630f1093 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:21.916155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b003a-77d5-4c6d-9c91-7e29630f1093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:21.934970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d24b003a-77d5-4c6d-9c91-7e29630f1093 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:21.936178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d24b003a-77d5-4c6d-9c91-7e29630f1093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:21.951084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:22.011351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 507/1022] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:21 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:22.033490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef591bc-af98-4ca8-9aee-14a2a0c96d31 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:22.143873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef591bc-af98-4ca8-9aee-14a2a0c96d31 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:22.179674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:22.180247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.180558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.180839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] PATH_INFO: `/v3/services/image` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.181474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:22.182848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:22.192348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:22.193531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:22.194433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:22.196829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 516/1023] 10.222.0.128 () {56 vars in 1110 bytes} [Tue Jul 26 15:17:22 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:22.200358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b003a-77d5-4c6d-9c91-7e29630f1093 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:22.307143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b003a-77d5-4c6d-9c91-7e29630f1093 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:22.350114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:22.350752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.351039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.351319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.352013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:22.352333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:22.353740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:22.364927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 508/1024] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:22 2022] GET /identity/v3/services?name=image => generated 123 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:22.366303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:22.434433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:22.480032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b1a80-9bc6-439b-98e4-db97ee53d89a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:22.518795 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97344ca-b757-4cf4-822d-6457b40975bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:22.519415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97344ca-b757-4cf4-822d-6457b40975bf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.519722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97344ca-b757-4cf4-822d-6457b40975bf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.520009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97344ca-b757-4cf4-822d-6457b40975bf None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.520751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97344ca-b757-4cf4-822d-6457b40975bf None admin] RBAC: Adding query filter params (type=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:22.521059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97344ca-b757-4cf4-822d-6457b40975bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:22.522451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97344ca-b757-4cf4-822d-6457b40975bf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:22.528571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 517/1025] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:22 2022] GET /identity/v3/services?type=image => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:22.534550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:22.551010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:22.603597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3fc0a-361e-4318-b831-a536fe4bd075 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.603881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3fc0a-361e-4318-b831-a536fe4bd075 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.604165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3fc0a-361e-4318-b831-a536fe4bd075 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.606613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 518/1026] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:22.611375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036923-e596-4192-be20-2c39f0924e97 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.611746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036923-e596-4192-be20-2c39f0924e97 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.612038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036923-e596-4192-be20-2c39f0924e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.630590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b5036923-e596-4192-be20-2c39f0924e97 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:22.631784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b5036923-e596-4192-be20-2c39f0924e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:22.654640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0552c3-39d6-4b10-8861-b81e46975505 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:22.692740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:22.693605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.694056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.694350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.695086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] RBAC: Adding query filter params (interface=public, service_id=dde5e0b7e546449c9c42c1803a00f274, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:22.695489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:22.696891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:22.704493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 509/1027] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:17:22 2022] GET /identity/v3/endpoints?service_id=dde5e0b7e546449c9c42c1803a00f274&interface=public®ion_id=RegionOne => generated 195 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:22.708172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:22.712498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 519/1028] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:22 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:22.733728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5036923-e596-4192-be20-2c39f0924e97 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:22.798116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:22.799217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:22.821379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb0707-b3df-4ae9-ad60-9a12d663cf24 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:22.843917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5036923-e596-4192-be20-2c39f0924e97 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:22.864180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:22.866592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.866884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.867164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.867840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:22.869510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:22.884512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 510/1029] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:17:22 2022] GET /identity/v3/services => generated 1966 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:17:22.893614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:22.894398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:22.895419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:22.895595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:22.921938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:22.927419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:22.928438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:22.929343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 520/1030] 10.222.0.128 () {56 vars in 1125 bytes} [Tue Jul 26 15:17:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:22.935003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.054840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6dd11-c8a1-4d7f-9544-3fc6867d4dcd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:23.095390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c90ed-4752-40c8-bd34-dcf44ce84c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:23.096023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c90ed-4752-40c8-bd34-dcf44ce84c61 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.096308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c90ed-4752-40c8-bd34-dcf44ce84c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.096618 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c90ed-4752-40c8-bd34-dcf44ce84c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.097661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c90ed-4752-40c8-bd34-dcf44ce84c61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:23.097900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c90ed-4752-40c8-bd34-dcf44ce84c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:23.099125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c90ed-4752-40c8-bd34-dcf44ce84c61 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:23.112665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 511/1031] 10.222.0.128 () {56 vars in 1120 bytes} [Tue Jul 26 15:17:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:23.114490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.223999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb1c12-be62-4ee7-9c04-83d7c38e6455 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:23.242700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:23.261086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:23.262507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.263862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.263862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.264046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cdf68-22c2-4ea3-9ddf-5452fbc3df93 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.264246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cdf68-22c2-4ea3-9ddf-5452fbc3df93 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.264981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cdf68-22c2-4ea3-9ddf-5452fbc3df93 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.266515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 512/1032] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:23 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:23.272936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457423-43f2-4515-8c3d-dbbc8d7f28a2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.273224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457423-43f2-4515-8c3d-dbbc8d7f28a2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.273506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457423-43f2-4515-8c3d-dbbc8d7f28a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.278823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:23.280608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:23.281292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:23.299734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e3457423-43f2-4515-8c3d-dbbc8d7f28a2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:23.300970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e3457423-43f2-4515-8c3d-dbbc8d7f28a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:23.301599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:23.302474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:23.303379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:23.303379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:23.303379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:23.303379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:23.303842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 521/1033] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:23.309272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.382626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 513/1034] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:23 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:23.408716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3457423-43f2-4515-8c3d-dbbc8d7f28a2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.425171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6d3d3-e02b-42c4-9da6-f7436203d16b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:23.468436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:23.469090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.469369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.469693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.470394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:23.470704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:23.472070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:23.489153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 522/1035] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:23.495127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.525733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:23.528853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3457423-43f2-4515-8c3d-dbbc8d7f28a2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:23.562756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:23.568677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:23.569344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.569628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.569931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.571136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:23.572606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:23.575342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 514/1036] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:23 2022] GET /identity/v3/domains/default => generated 200 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:23.581767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.609497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb2d34-a748-4a08-8e23-0537c8ef5433 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:23.651607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:23.652005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.652005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.652897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.661069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:23.662532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:17:23.676559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:23.677592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:23.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 523/1037] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:17:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:23.684477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.700587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3e16-7150-4eae-8114-0d25105bf2e7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:23.738270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:23.738894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.739180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.739460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.740149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:23.741620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:23.792420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:23.802484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bc14e-99e3-4207-97b4-1f5017248dbb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:23.836350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:23.836913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:23.837196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:23.837472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:23.838265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:23.838573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:23.840206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:23.858532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 524/1038] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:17:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:23.860669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 515/1039] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:17:23 2022] POST /identity/v3/users => generated 286 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:23.865131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:23.985234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4760424-3473-49ce-a9c3-d87c03b357d6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:24.026382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:24.027572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:24.027862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:24.028142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:24.048821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:24.108416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:24.109897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:24.212607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 525/1040] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:23 2022] GET /identity/v3/auth/tokens => generated 1963 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:17:24.219019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:24.278269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:24.324794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943018-5477-4a6d-b8e2-ac5024923c07 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:24.360296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:24.360893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:24.361177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:24.361452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:24.373104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:24.374555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:24.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:24.390942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:24.391849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:24.391849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:24.391849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:24.391849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:24.391849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 516/1041] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:24.398157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:24.453042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:24.509028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1426c61-21b6-4bce-8b7c-84917754cf8d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:24.544799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:24.545427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:24.545709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:24.545985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:24.546680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:24.546989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:24.548357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:24.560418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 526/1042] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:24.565845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:24.675125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d4934-6f58-45fd-bab5-e32647496b3d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:24.684848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:24.710539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:24.711186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:24.711475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:24.711755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:24.718611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:24.720556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:17:24.733420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:24.734544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:24.735534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:24.738375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 517/1043] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:24.744009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:24.858272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db53603-6c40-4de6-8952-d645e5d9615b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:24.867514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:24.894274 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e4b98-f695-4365-bc6d-56bc97babafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:24.894905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e4b98-f695-4365-bc6d-56bc97babafd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:24.895192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e4b98-f695-4365-bc6d-56bc97babafd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:24.895476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e4b98-f695-4365-bc6d-56bc97babafd None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:24.896237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e4b98-f695-4365-bc6d-56bc97babafd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:24.896578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e4b98-f695-4365-bc6d-56bc97babafd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:24.898249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e4b98-f695-4365-bc6d-56bc97babafd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:24.916133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 527/1044] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:24.917476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:25.025489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467784-f38a-4cfd-973e-bf3f502ff471 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:25.031173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927bc4f-4e78-4582-85db-47cfd33ee98c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.031658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927bc4f-4e78-4582-85db-47cfd33ee98c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.031762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927bc4f-4e78-4582-85db-47cfd33ee98c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.033414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 528/1045] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:25.038508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779dec0-2662-4730-a15e-27324bdf827c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.038793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779dec0-2662-4730-a15e-27324bdf827c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.039069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779dec0-2662-4730-a15e-27324bdf827c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.057171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a779dec0-2662-4730-a15e-27324bdf827c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:25.059220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a779dec0-2662-4730-a15e-27324bdf827c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:25.068158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:25.069157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.069588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.070031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.071130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] RBAC: Adding query filter params (role.id=e08dc961ff534e868714c89db2909627, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=92e40a33db1a41d59d9a7a3bef5f3ece) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:25.071623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:25.073411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:25.081195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 518/1046] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:24 2022] GET /identity/v3/role_assignments?user.id=92e40a33db1a41d59d9a7a3bef5f3ece&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=e08dc961ff534e868714c89db2909627 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:25.146839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 529/1047] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:25 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:25.170573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:25.206450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:25.279894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ef35d-6c29-40b3-8dfb-14e642822e2d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:25.315032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:25.315680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.315970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.316247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.316909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:25.318284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:25.328085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:25.329461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:25.335183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 519/1048] 10.222.0.128 () {56 vars in 1110 bytes} [Tue Jul 26 15:17:25 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:25.337597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779dec0-2662-4730-a15e-27324bdf827c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:25.438042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:25.442390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779dec0-2662-4730-a15e-27324bdf827c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:25.485645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:25.486273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.486555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.486831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.487521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:25.488092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:25.489501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:25.499591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 530/1049] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:25 2022] GET /identity/v3/services?name=image => generated 123 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:25.501436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:25.602497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:25.608887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cea0f4-c011-462a-8ac8-0d8e3ca3abb6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:25.644559 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d6b3c-f96f-4f78-95c7-7455402229f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:25.645186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d6b3c-f96f-4f78-95c7-7455402229f0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.645468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d6b3c-f96f-4f78-95c7-7455402229f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.645802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d6b3c-f96f-4f78-95c7-7455402229f0 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.646459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d6b3c-f96f-4f78-95c7-7455402229f0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:25.646763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d6b3c-f96f-4f78-95c7-7455402229f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:25.648131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d6b3c-f96f-4f78-95c7-7455402229f0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:25.658219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 520/1050] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:25 2022] GET /identity/v3/services?type=image => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:25.659523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:25.765547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d3fc2-add4-4ac0-8878-5b48512f85e3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:25.801402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f4c36-dab5-421e-9c4d-328bf7aae868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:25.802027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f4c36-dab5-421e-9c4d-328bf7aae868 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.802307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f4c36-dab5-421e-9c4d-328bf7aae868 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.802582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f4c36-dab5-421e-9c4d-328bf7aae868 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.803293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f4c36-dab5-421e-9c4d-328bf7aae868 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:25.804698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f4c36-dab5-421e-9c4d-328bf7aae868 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:25.940580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 531/1051] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:17:25 2022] POST /identity/v3/endpoints => generated 329 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:25.992988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c271cfe-4da5-4b0b-9365-bf2ecb93d87a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:25.993364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c271cfe-4da5-4b0b-9365-bf2ecb93d87a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:25.993720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c271cfe-4da5-4b0b-9365-bf2ecb93d87a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:25.995476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 521/1052] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:26.001588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab302e6-0f77-4ca3-8190-0dc288a9bcc5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:26.001893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab302e6-0f77-4ca3-8190-0dc288a9bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:26.002152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab302e6-0f77-4ca3-8190-0dc288a9bcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:26.022052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8ab302e6-0f77-4ca3-8190-0dc288a9bcc5 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:26.023297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8ab302e6-0f77-4ca3-8190-0dc288a9bcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:26.040526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:26.114369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 532/1053] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:25 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:26.138375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c271cfe-4da5-4b0b-9365-bf2ecb93d87a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:26.257828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c271cfe-4da5-4b0b-9365-bf2ecb93d87a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:26.293868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:26.294505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:26.294723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:26.294997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:26.306488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:26.312004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:26.313032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:26.313911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:26.315202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 522/1054] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:17:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:26.319361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab302e6-0f77-4ca3-8190-0dc288a9bcc5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:26.423331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab302e6-0f77-4ca3-8190-0dc288a9bcc5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:26.466991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:26.467622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:26.467909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:26.468184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:26.468948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:26.469261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:26.470634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:26.479520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 533/1055] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:17:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:26.485417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:26.509574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:26.593808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9c605-ea15-4af6-becd-016a3b59febe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:26.640323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:26.641019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:26.641307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:26.641588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] PATH_INFO: `/v3/users/ad7962a8b7fe461c9844a684e15ee924` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:26.656636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] RBAC: Authorizing `identity:get_user(user_id=ad7962a8b7fe461c9844a684e15ee924)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:26.658103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:26.664480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 523/1056] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:26 2022] GET /identity/v3/users/ad7962a8b7fe461c9844a684e15ee924 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:26.665715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:26.676234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:26.781792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab3349-ff65-4136-87e9-b7473385489e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:26.822108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90aca8-372a-4fa4-bd04-d507c2f672a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:26.822749 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90aca8-372a-4fa4-bd04-d507c2f672a1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:26.823033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90aca8-372a-4fa4-bd04-d507c2f672a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:26.823313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90aca8-372a-4fa4-bd04-d507c2f672a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:26.909235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90aca8-372a-4fa4-bd04-d507c2f672a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:26.910708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90aca8-372a-4fa4-bd04-d507c2f672a1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:26.923899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:27.024507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 534/1057] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:26 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 15:17:27.030087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:27.117966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:27.140667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27766c78-0377-4f1e-9436-b05df019656e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.141206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27766c78-0377-4f1e-9436-b05df019656e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.141206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27766c78-0377-4f1e-9436-b05df019656e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.142606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 535/1058] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:27.144315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6153fc-5d40-4064-923c-7fc4511b9e4a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:27.148229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde74cf-7850-4b9a-911f-4091550d01e6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.148659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde74cf-7850-4b9a-911f-4091550d01e6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.149121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde74cf-7850-4b9a-911f-4091550d01e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.168540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5fde74cf-7850-4b9a-911f-4091550d01e6 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:27.170071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5fde74cf-7850-4b9a-911f-4091550d01e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:27.183712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:27.184363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.184677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.184962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.186140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] RBAC: Authorizing `identity:get_project(project_id=d41d2a892a1445c28bdaa82a011d4ddd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:27.187577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:27.194176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 524/1059] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:27 2022] GET /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:27.195417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:27.246180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 536/1060] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:27 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:27.268488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde74cf-7850-4b9a-911f-4091550d01e6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:27.308745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e966098-a9bc-4acb-a3be-93754a66c8ca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:27.310711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:27.349453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:27.350079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.350361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.350642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.351438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd, user.id=ad7962a8b7fe461c9844a684e15ee924) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:27.351755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:27.353207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:27.364504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 525/1061] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:27 2022] GET /identity/v3/role_assignments?user.id=ad7962a8b7fe461c9844a684e15ee924&scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:27.395321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde74cf-7850-4b9a-911f-4091550d01e6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:27.436517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:27.437246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.437534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.437808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.453296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:27.458742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:27.459763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:27.460642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:27.463362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 537/1062] 10.222.0.128 () {56 vars in 1125 bytes} [Tue Jul 26 15:17:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:27.469457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:27.483957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:27.581648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea4e21-068b-4698-be84-68a34de79ad5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:27.617096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:27.617720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.618003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.618280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.619000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:27.619322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:27.622007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:27.635177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 526/1063] 10.222.0.128 () {56 vars in 1120 bytes} [Tue Jul 26 15:17:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:27.636443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:27.734584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:27.741139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5877f-c569-46c9-8f86-abc3be857d0d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:27.779230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:27.779880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.780160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.780433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.791075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:27.793145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:27.809233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:27.810202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:27.811108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:27.811108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:27.811108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:27.811108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:27.811108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 538/1064] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:27.816496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:27.890714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:27.923834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17305f7-ef1e-47b7-bcf0-829afd5a5382 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:27.958473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27208870-d9b2-403e-9605-4b077727e553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:27.959092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208870-d9b2-403e-9605-4b077727e553 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:27.959382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208870-d9b2-403e-9605-4b077727e553 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:27.959656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208870-d9b2-403e-9605-4b077727e553 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:27.960347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27208870-d9b2-403e-9605-4b077727e553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:27.960685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27208870-d9b2-403e-9605-4b077727e553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:27.962044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27208870-d9b2-403e-9605-4b077727e553 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:27.979749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 527/1065] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:27.981747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.078013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549636-5d64-446a-8973-8db514e9e90a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.078259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549636-5d64-446a-8973-8db514e9e90a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.078508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549636-5d64-446a-8973-8db514e9e90a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.079940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 528/1066] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:28.085548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654432b-26c1-432a-a547-94204a5f0306 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.085736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baddbb7-3db8-4db3-9000-7eaf34e8ba59 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.085824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654432b-26c1-432a-a547-94204a5f0306 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.086111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654432b-26c1-432a-a547-94204a5f0306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.106983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b654432b-26c1-432a-a547-94204a5f0306 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:28.108195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b654432b-26c1-432a-a547-94204a5f0306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:28.124354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.125041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.125325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.125666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.133648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.135092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:17:28.147665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:28.149332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:28.150280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 539/1067] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:17:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:28.155157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.166223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:28.188686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 529/1068] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:28 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:28.208539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654432b-26c1-432a-a547-94204a5f0306 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.264480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d05ca-09ee-4652-9dc1-96872b05885c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.275131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:28.305008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.305629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.305911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.306186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.306982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:28.307511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.309741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.321227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654432b-26c1-432a-a547-94204a5f0306 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.331622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 540/1069] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:28.337033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.360731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.361359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.361645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.361923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.412403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:28.439089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.440648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.450870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8ec5e-a30d-4c6c-9a60-4f67243c1703 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.493862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.494508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.494796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.495071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.507372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.508034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:28.509894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:28.527048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:28.533932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:28.534833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:28.534833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:28.534833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:28.534833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:28.534833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 541/1070] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:28.534833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.562109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 530/1071] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 15:17:28.568098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.646858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4f8c0-8bfe-4e61-b121-f54244e4e9d4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.666355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:28.685191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.685826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.686114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.686397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.687104 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:28.687374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7d13f-6eeb-466d-92d1-6e8ddf660085 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.687503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.688807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.704126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 542/1072] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:28.709572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.732404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.733085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.733369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.733653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.745424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.747312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:28.764692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:28.765673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:28.766663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:28.766663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:28.766663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:28.766663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:28.766663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 531/1073] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:28.771783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.789457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:28.825987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6a459-50aa-491f-8629-8a7d552cafdb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.862874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.863531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.863816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.864094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.871793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.873337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.884325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf0785-9cf4-49e2-831d-32d8daf82bd3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:17:28.885204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:28.886726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:28.887630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 543/1074] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:28.892762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:28.926693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:28.927385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:28.927728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:28.928012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:28.928768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:28.929194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:28.930466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:28.936691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:28.944019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 532/1075] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:29.009237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2cd9c-6530-4625-880b-c782f8d5d193 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:29.048002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63009336-a55c-4aee-bb08-5faea1db4ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:29.049567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63009336-a55c-4aee-bb08-5faea1db4ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:29.049567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63009336-a55c-4aee-bb08-5faea1db4ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:29.049567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63009336-a55c-4aee-bb08-5faea1db4ab5 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:29.050045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63009336-a55c-4aee-bb08-5faea1db4ab5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:29.050910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63009336-a55c-4aee-bb08-5faea1db4ab5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:29.051970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63009336-a55c-4aee-bb08-5faea1db4ab5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:29.072736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 544/1076] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:29.074621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:29.161159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:29.191589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3d5d-bce5-4ff5-83d3-4161575bbcea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:29.220672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77508fa6-ee41-440c-ab53-03ad895794de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:29.221268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77508fa6-ee41-440c-ab53-03ad895794de None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:29.221550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77508fa6-ee41-440c-ab53-03ad895794de None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:29.221829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77508fa6-ee41-440c-ab53-03ad895794de None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/92e40a33db1a41d59d9a7a3bef5f3ece/roles/e08dc961ff534e868714c89db2909627` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:29.224008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77508fa6-ee41-440c-ab53-03ad895794de None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=92e40a33db1a41d59d9a7a3bef5f3ece, role_id=e08dc961ff534e868714c89db2909627)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:29.225467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77508fa6-ee41-440c-ab53-03ad895794de None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:29.314451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 533/1077] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:17:29 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/92e40a33db1a41d59d9a7a3bef5f3ece/roles/e08dc961ff534e868714c89db2909627 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:29.462947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45457b4-b7fd-4240-adfd-4d46b9a0a0c7 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:29.463397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45457b4-b7fd-4240-adfd-4d46b9a0a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:29.463494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45457b4-b7fd-4240-adfd-4d46b9a0a0c7 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:29.465543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 545/1078] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:29 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:29.471471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b3a05-bd1a-453e-9900-8b8acab70bf4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:29.471952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b3a05-bd1a-453e-9900-8b8acab70bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:29.471952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b3a05-bd1a-453e-9900-8b8acab70bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:29.496363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2f9b3a05-bd1a-453e-9900-8b8acab70bf4 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:29.498322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f9b3a05-bd1a-453e-9900-8b8acab70bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:29.581901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 534/1079] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:29 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:29.605887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45457b4-b7fd-4240-adfd-4d46b9a0a0c7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:29.705880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:29.710892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45457b4-b7fd-4240-adfd-4d46b9a0a0c7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:29.747315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:29.747881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:29.748186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:29.748574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:29.758799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:29.763938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:29.764932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:29.765801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 546/1080] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:29.771380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b3a05-bd1a-453e-9900-8b8acab70bf4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:29.834565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:29.874773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b3a05-bd1a-453e-9900-8b8acab70bf4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:29.911122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:29.911750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:29.912037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:29.912318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:29.913098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:29.913413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:29.914827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:29.923729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 535/1081] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:17:29 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:29.929573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:30.040978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70272706-399a-4679-ace9-d5be92229d47 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:30.075512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:30.076150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:30.076435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:30.076743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] PATH_INFO: `/v3/users/ad7962a8b7fe461c9844a684e15ee924` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:30.077912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] RBAC: Authorizing `identity:get_user(user_id=ad7962a8b7fe461c9844a684e15ee924)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:30.079348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:30.081002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 547/1082] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:29 2022] GET /identity/v3/users/ad7962a8b7fe461c9844a684e15ee924 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:30.086868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:30.188629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5c365-a5e2-4cbe-aed7-e81b2c83f7bd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:30.215111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:30.224492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce9e5a-0d9b-4cf1-a965-19bd5983f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:30.225151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9e5a-0d9b-4cf1-a965-19bd5983f9ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:30.225468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9e5a-0d9b-4cf1-a965-19bd5983f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:30.225781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9e5a-0d9b-4cf1-a965-19bd5983f9ff None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:30.226943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce9e5a-0d9b-4cf1-a965-19bd5983f9ff None admin] RBAC: Authorizing `identity:get_project(project_id=d41d2a892a1445c28bdaa82a011d4ddd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:30.228400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce9e5a-0d9b-4cf1-a965-19bd5983f9ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:30.230658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 536/1083] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:30 2022] GET /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:30.236601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:30.327929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:30.403213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb623552-dbd1-4f8c-9f51-6f743aef7e23 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:30.438550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f75ab1-f23a-4def-8f17-fb165686e6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:30.439253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f75ab1-f23a-4def-8f17-fb165686e6ee None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:30.439536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f75ab1-f23a-4def-8f17-fb165686e6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:30.439811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f75ab1-f23a-4def-8f17-fb165686e6ee None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd/users/ad7962a8b7fe461c9844a684e15ee924/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:30.442003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f75ab1-f23a-4def-8f17-fb165686e6ee None admin] RBAC: Authorizing `identity:create_grant(project_id=d41d2a892a1445c28bdaa82a011d4ddd, user_id=ad7962a8b7fe461c9844a684e15ee924, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:30.443486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f75ab1-f23a-4def-8f17-fb165686e6ee None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:30.510370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 548/1084] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:17:30 2022] PUT /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd/users/ad7962a8b7fe461c9844a684e15ee924/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:17:31.348374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9b3d0-6afd-4527-993f-59dae42da389 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:31.348931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9b3d0-6afd-4527-993f-59dae42da389 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:31.348931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9b3d0-6afd-4527-993f-59dae42da389 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:31.350297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 537/1085] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:31.356212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621db42-122e-4710-abae-588d8bccb6bf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:31.356528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621db42-122e-4710-abae-588d8bccb6bf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:31.356811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621db42-122e-4710-abae-588d8bccb6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:31.375598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6621db42-122e-4710-abae-588d8bccb6bf None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:31.376812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6621db42-122e-4710-abae-588d8bccb6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:31.443604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 549/1086] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:31 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:31.466404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9b3d0-6afd-4527-993f-59dae42da389 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:31.569940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9b3d0-6afd-4527-993f-59dae42da389 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:31.603954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:31.604614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:31.604901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:31.605179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:31.617111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:31.622815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:31.624002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:31.625164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 538/1087] 10.222.0.128 () {56 vars in 1125 bytes} [Tue Jul 26 15:17:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:31.630402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621db42-122e-4710-abae-588d8bccb6bf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:31.648428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:31.731009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621db42-122e-4710-abae-588d8bccb6bf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:31.768796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:31.769425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:31.769712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:31.769992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:31.770717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:31.771029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:31.772401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:31.780549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 550/1088] 10.222.0.128 () {56 vars in 1120 bytes} [Tue Jul 26 15:17:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:31.786285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:31.841683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:31.887056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5077bd-6ea4-47a9-aa68-d2c6372ee9e7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:31.923358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:31.924004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:31.924292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:31.924604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:31.935698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:31.937182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:31.952009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:31.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:31.954113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:31.954113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:31.954113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:31.954113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:31.954113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 539/1089] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:31.959015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.040117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:32.062507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ad41b-3869-40cb-b414-50c9d44d1208 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:32.097942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:32.098570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.098854 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.099129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.099826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:32.100136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:32.101536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:32.115359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 551/1090] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:32.121233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.222768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b1e49-46ac-4cdf-8064-1c9e854b33e4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:32.229790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:32.255762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:32.256396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.256796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.257068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.265064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:32.266485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:17:32.279030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:32.280183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 15:17:32.281204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:32.284617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 540/1091] 10.222.0.128 () {56 vars in 1098 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:32.286094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.389455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf3e3e-c49b-4cb6-a433-8d92f595516c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:32.425380 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc9046-7d7e-43ea-ad9d-b47c95e5c08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:32.426014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9046-7d7e-43ea-ad9d-b47c95e5c08b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.426300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9046-7d7e-43ea-ad9d-b47c95e5c08b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.426582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9046-7d7e-43ea-ad9d-b47c95e5c08b None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.427385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc9046-7d7e-43ea-ad9d-b47c95e5c08b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:32.427698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc9046-7d7e-43ea-ad9d-b47c95e5c08b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:32.429172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc9046-7d7e-43ea-ad9d-b47c95e5c08b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:32.437771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:32.452744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 552/1092] 10.222.0.128 () {56 vars in 1138 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:32.454463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.536071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115e081-d2f7-4e2c-9170-5afcbabaa0e8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.536257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115e081-d2f7-4e2c-9170-5afcbabaa0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.536674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115e081-d2f7-4e2c-9170-5afcbabaa0e8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.538045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 553/1093] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:32.543095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cb6d2-87e1-47a7-aa93-e7cf4e116ecf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.543385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cb6d2-87e1-47a7-aa93-e7cf4e116ecf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.543667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cb6d2-87e1-47a7-aa93-e7cf4e116ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.561437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fea50-3422-4660-af28-8311088d5f90 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:32.562715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-369cb6d2-87e1-47a7-aa93-e7cf4e116ecf None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:32.563915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-369cb6d2-87e1-47a7-aa93-e7cf4e116ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:32.596309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:32.596962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.597248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.597528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.631942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:32.632132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 554/1094] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:32 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:32.656729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cb6d2-87e1-47a7-aa93-e7cf4e116ecf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.672984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:32.674707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:32.758503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cb6d2-87e1-47a7-aa93-e7cf4e116ecf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:32.777154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 541/1095] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:17:32.783102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.788238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:32.797666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:32.798320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.799757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.799757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.809822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:32.815001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:32.816017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:32.817030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 555/1096] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:32.821979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.849066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:32.885014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe6fc-c6db-4de4-8be6-5b58240c24d8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:32.919752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:32.920395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.920899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.921014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.925723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3658-fb2c-4c53-b5be-4d6e83d06a1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:32.931002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:32.933097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:17:32.948157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:32.949446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:32.950791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:17:32.950791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:17:32.950791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:17:32.950791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:32.950791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 542/1097] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:32.954972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:32.960491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:32.961109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:32.961388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:32.961661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:32.962382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:32.962688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:32.964177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:32.973291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 556/1098] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:32.978844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:33.030124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:33.058207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f96fde-56eb-4d5e-8798-031d78e56cff None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.075929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:33.083806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8765c8-0a17-4762-9aa7-5d8a6268a88a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.096593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.097231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.097514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.097790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.098492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:33.098803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.100165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.113226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 543/1099] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:33.118815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:33.125510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.126157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.126444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.126723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] PATH_INFO: `/v3/users/ad7962a8b7fe461c9844a684e15ee924` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.127916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] RBAC: Authorizing `identity:get_user(user_id=ad7962a8b7fe461c9844a684e15ee924)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.129386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.131006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 557/1100] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:32 2022] GET /identity/v3/users/ad7962a8b7fe461c9844a684e15ee924 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:33.136959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:33.225704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612119a-e183-40f5-8fb6-e34cafd0922a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.239835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd409c0-9159-475d-9b1f-ddb3e6fc709c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.256601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:33.262222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.262885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.263175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.263455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.270406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.276606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.276721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.276721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.276721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.276721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:17:33.283271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:33.284581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:33.285528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 544/1101] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:17:33.290365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:33.331607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:33.349356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.350813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.392415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fabf6-248b-4f1d-8208-06f4a1c04db3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.425963 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.426586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.426867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.427141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.428102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:33.428413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.429869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.443601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 545/1102] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:33.449802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:33.458727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 558/1103] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:33 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:17:33.464591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:33.482850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:33.551979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:33.555896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459a936-d78e-419c-9dcb-42971477c8ec None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.565532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2187bb-a01d-49ac-aa15-1e8960adfb17 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.592984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.593527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.593811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.594085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.594877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] RBAC: Adding query filter params (role.id=e08dc961ff534e868714c89db2909627, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=92e40a33db1a41d59d9a7a3bef5f3ece) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:33.595191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.596640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.600405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5e138-5382-46c3-85f6-9f8f0104892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.601074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5e138-5382-46c3-85f6-9f8f0104892b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.601354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5e138-5382-46c3-85f6-9f8f0104892b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.601631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5e138-5382-46c3-85f6-9f8f0104892b None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.602710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 546/1104] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:33 2022] GET /identity/v3/role_assignments?user.id=92e40a33db1a41d59d9a7a3bef5f3ece&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=e08dc961ff534e868714c89db2909627 => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:33.602828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5e138-5382-46c3-85f6-9f8f0104892b None admin] RBAC: Authorizing `identity:get_project(project_id=d41d2a892a1445c28bdaa82a011d4ddd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.604934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5e138-5382-46c3-85f6-9f8f0104892b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.606680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 559/1105] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:33 2022] GET /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:33.613188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:33.720967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0e678-429d-4faa-ba69-9c9a4f456420 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:33.758771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2be507-e92e-470b-8bd9-1093e0e5b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:33.759452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be507-e92e-470b-8bd9-1093e0e5b4fa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:33.759681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be507-e92e-470b-8bd9-1093e0e5b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:33.760068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be507-e92e-470b-8bd9-1093e0e5b4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:33.760888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2be507-e92e-470b-8bd9-1093e0e5b4fa None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd, user.id=ad7962a8b7fe461c9844a684e15ee924) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:33.761212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2be507-e92e-470b-8bd9-1093e0e5b4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:33.762835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2be507-e92e-470b-8bd9-1093e0e5b4fa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:33.769143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 547/1106] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:33 2022] GET /identity/v3/role_assignments?user.id=ad7962a8b7fe461c9844a684e15ee924&scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:35.844556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121975c3-54fe-4c3d-955c-052a0b11b37c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:35.845060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121975c3-54fe-4c3d-955c-052a0b11b37c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:35.845060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121975c3-54fe-4c3d-955c-052a0b11b37c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:35.846392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 560/1107] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:35.851518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d30b-90cd-4900-87b3-dacd091f835e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:35.851925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d30b-90cd-4900-87b3-dacd091f835e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:35.852077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d30b-90cd-4900-87b3-dacd091f835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:35.870090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d366d30b-90cd-4900-87b3-dacd091f835e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:35.871280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d366d30b-90cd-4900-87b3-dacd091f835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:35.891795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:35.941988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 548/1108] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:35 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:35.965307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121975c3-54fe-4c3d-955c-052a0b11b37c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:36.050622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:36.067871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121975c3-54fe-4c3d-955c-052a0b11b37c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:36.104648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59686d8-d35f-4844-90a4-4c0f7d9143f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:36.104991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59686d8-d35f-4844-90a4-4c0f7d9143f4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:36.105278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59686d8-d35f-4844-90a4-4c0f7d9143f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:36.105548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59686d8-d35f-4844-90a4-4c0f7d9143f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:36.106750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59686d8-d35f-4844-90a4-4c0f7d9143f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:36.108175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59686d8-d35f-4844-90a4-4c0f7d9143f4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:36.109792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 561/1109] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:35 2022] GET /identity/v3/domains/default => generated 200 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:36.115965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366d30b-90cd-4900-87b3-dacd091f835e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:36.215948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366d30b-90cd-4900-87b3-dacd091f835e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:36.249977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c6d24-87ce-4267-b4e7-eec2c4ef8a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:36.250616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c6d24-87ce-4267-b4e7-eec2c4ef8a96 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:36.250880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c6d24-87ce-4267-b4e7-eec2c4ef8a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:36.251159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c6d24-87ce-4267-b4e7-eec2c4ef8a96 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:36.251847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c6d24-87ce-4267-b4e7-eec2c4ef8a96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:36.253303 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c6d24-87ce-4267-b4e7-eec2c4ef8a96 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:36.366937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 549/1110] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:17:36 2022] POST /identity/v3/users => generated 287 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:38.456941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538c312-17f0-4a0b-b095-50205fc6b6a9 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:38.457467 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538c312-17f0-4a0b-b095-50205fc6b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:38.457554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538c312-17f0-4a0b-b095-50205fc6b6a9 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:38.458863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 562/1111] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:38.465257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea1086-2e97-4b8b-a8dc-ded0c22522c9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:38.465707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea1086-2e97-4b8b-a8dc-ded0c22522c9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:38.465803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea1086-2e97-4b8b-a8dc-ded0c22522c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:38.484483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-42ea1086-2e97-4b8b-a8dc-ded0c22522c9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:38.485682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-42ea1086-2e97-4b8b-a8dc-ded0c22522c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:38.493254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:38.553654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 550/1112] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:38 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:38.577478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538c312-17f0-4a0b-b095-50205fc6b6a9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:38.677944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538c312-17f0-4a0b-b095-50205fc6b6a9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:38.712539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:38.713235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:38.714031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:38.714308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] PATH_INFO: `/v3/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:38.716001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] RBAC: Authorizing `identity:get_role(role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:38.717498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:38.718888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 563/1113] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:38 2022] GET /identity/v3/roles/364a95d0748e48e9a538b4e6be293f91 => generated 224 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:38.724793 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea1086-2e97-4b8b-a8dc-ded0c22522c9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:38.826004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea1086-2e97-4b8b-a8dc-ded0c22522c9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:38.863287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:38.863923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:38.864204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:38.864504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] PATH_INFO: `/v3/users/3b4dca35d9bb4f8886d58d2b169551c5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:38.875662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:38.877920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] RBAC: Authorizing `identity:get_user(user_id=3b4dca35d9bb4f8886d58d2b169551c5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:38.879358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:38.881013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 551/1114] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:38 2022] GET /identity/v3/users/3b4dca35d9bb4f8886d58d2b169551c5 => generated 287 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:38.887748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:38.910115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:38.989870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02d049-176d-441c-8fbf-b1ad7a5319a5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:39.024515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:39.025123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:39.025401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:39.025683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:39.092287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:39.093771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:39.114022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:39.194235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 564/1115] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:38 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:17:39.200924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:39.305587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6444ff3-3774-4618-8725-98f8af8cf70e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:39.340819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6df952-1b65-4e77-9126-b3cf4ce887b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:39.341505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6df952-1b65-4e77-9126-b3cf4ce887b3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:39.341789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6df952-1b65-4e77-9126-b3cf4ce887b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:39.342065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6df952-1b65-4e77-9126-b3cf4ce887b3 None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:39.343233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6df952-1b65-4e77-9126-b3cf4ce887b3 None admin] RBAC: Authorizing `identity:get_project(project_id=d41d2a892a1445c28bdaa82a011d4ddd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:39.344690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6df952-1b65-4e77-9126-b3cf4ce887b3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:39.346519 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 552/1116] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:39 2022] GET /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:39.352730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:39.452727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade66b69-7027-4b9a-9e5b-36ccc30a93da None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:39.473755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:39.487985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddf6b4-d982-4ffd-bd89-851565f26694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:39.488633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddf6b4-d982-4ffd-bd89-851565f26694 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:39.488913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddf6b4-d982-4ffd-bd89-851565f26694 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:39.489188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddf6b4-d982-4ffd-bd89-851565f26694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:39.489968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddf6b4-d982-4ffd-bd89-851565f26694 None admin] RBAC: Adding query filter params (role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd, user.id=3b4dca35d9bb4f8886d58d2b169551c5) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:39.490277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddf6b4-d982-4ffd-bd89-851565f26694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:39.491696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddf6b4-d982-4ffd-bd89-851565f26694 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:39.497738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 565/1117] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:39 2022] GET /identity/v3/role_assignments?user.id=3b4dca35d9bb4f8886d58d2b169551c5&scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 260 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:41.495438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b166-2f86-40eb-a2f3-3c28c386965a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:41.495917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b166-2f86-40eb-a2f3-3c28c386965a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:41.496035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b166-2f86-40eb-a2f3-3c28c386965a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:41.497367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 553/1118] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:41.502960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e198-48b8-4167-bc6f-9457418d8648 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:41.503343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e198-48b8-4167-bc6f-9457418d8648 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:41.503509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e198-48b8-4167-bc6f-9457418d8648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:41.521185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9589e198-48b8-4167-bc6f-9457418d8648 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:41.522371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9589e198-48b8-4167-bc6f-9457418d8648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:41.588562 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 566/1119] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:41 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:41.610943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759b166-2f86-40eb-a2f3-3c28c386965a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:41.679539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:41.715603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759b166-2f86-40eb-a2f3-3c28c386965a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:41.750674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:41.751311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:41.751595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:41.751869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] PATH_INFO: `/v3/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:41.753350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] RBAC: Authorizing `identity:get_role(role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:41.754724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:41.756056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 554/1120] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:41 2022] GET /identity/v3/roles/364a95d0748e48e9a538b4e6be293f91 => generated 224 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:41.762273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589e198-48b8-4167-bc6f-9457418d8648 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:41.859405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589e198-48b8-4167-bc6f-9457418d8648 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:41.874052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:41.895555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:41.896204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:41.896509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:41.896792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] PATH_INFO: `/v3/users/3b4dca35d9bb4f8886d58d2b169551c5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:41.899119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] RBAC: Authorizing `identity:get_user(user_id=3b4dca35d9bb4f8886d58d2b169551c5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:41.900754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:41.902424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 567/1121] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:41 2022] GET /identity/v3/users/3b4dca35d9bb4f8886d58d2b169551c5 => generated 287 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:41.908284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:42.015709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160bfa-a750-4504-9ee0-588af48ed715 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:42.051254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911912c7-9170-46d4-a319-5c0152549313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:42.052078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911912c7-9170-46d4-a319-5c0152549313 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:42.052440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911912c7-9170-46d4-a319-5c0152549313 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:42.052747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911912c7-9170-46d4-a319-5c0152549313 None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:42.054298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911912c7-9170-46d4-a319-5c0152549313 None admin] RBAC: Authorizing `identity:get_project(project_id=d41d2a892a1445c28bdaa82a011d4ddd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:42.056099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911912c7-9170-46d4-a319-5c0152549313 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:42.058110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 555/1122] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:41 2022] GET /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:42.064233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:42.162512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0b348-7c73-4ab3-893d-1bd6d8f42f91 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:42.197456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cabb5-1c4d-4ea5-a05e-46b40f2f9c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:42.198082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cabb5-1c4d-4ea5-a05e-46b40f2f9c6d None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:42.198359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cabb5-1c4d-4ea5-a05e-46b40f2f9c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:42.198632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cabb5-1c4d-4ea5-a05e-46b40f2f9c6d None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd/users/3b4dca35d9bb4f8886d58d2b169551c5/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:42.200806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cabb5-1c4d-4ea5-a05e-46b40f2f9c6d None admin] RBAC: Authorizing `identity:create_grant(project_id=d41d2a892a1445c28bdaa82a011d4ddd, user_id=3b4dca35d9bb4f8886d58d2b169551c5, role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:42.202227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cabb5-1c4d-4ea5-a05e-46b40f2f9c6d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:42.272721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 568/1123] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:17:42 2022] PUT /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd/users/3b4dca35d9bb4f8886d58d2b169551c5/roles/364a95d0748e48e9a538b4e6be293f91 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:44.307731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e02184-653b-46af-811f-792bfe1bf950 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:44.308262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e02184-653b-46af-811f-792bfe1bf950 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:44.308383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e02184-653b-46af-811f-792bfe1bf950 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:44.309672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 556/1124] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:17:44.315986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54618b-878b-4236-aea4-26031b5e40de None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:44.316449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54618b-878b-4236-aea4-26031b5e40de None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:44.316582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54618b-878b-4236-aea4-26031b5e40de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:44.334707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4f54618b-878b-4236-aea4-26031b5e40de None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:44.335900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4f54618b-878b-4236-aea4-26031b5e40de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:44.372905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:44.406251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 569/1125] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:44 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:44.430563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e02184-653b-46af-811f-792bfe1bf950 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:44.434922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:44.535386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e02184-653b-46af-811f-792bfe1bf950 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:44.569480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:44.570033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:44.570315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:44.570590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] PATH_INFO: `/v3/roles/364a95d0748e48e9a538b4e6be293f91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:44.572012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] RBAC: Authorizing `identity:get_role(role_id=364a95d0748e48e9a538b4e6be293f91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:44.573444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:44.574768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 557/1126] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:44 2022] GET /identity/v3/roles/364a95d0748e48e9a538b4e6be293f91 => generated 224 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:44.580863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54618b-878b-4236-aea4-26031b5e40de None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:44.682497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54618b-878b-4236-aea4-26031b5e40de None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:44.729710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:44.730358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:44.730643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:44.730922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] PATH_INFO: `/v3/users/3b4dca35d9bb4f8886d58d2b169551c5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:44.732199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] RBAC: Authorizing `identity:get_user(user_id=3b4dca35d9bb4f8886d58d2b169551c5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:44.733679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:44.735309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 570/1127] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:44 2022] GET /identity/v3/users/3b4dca35d9bb4f8886d58d2b169551c5 => generated 287 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:44.742037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:44.821076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:44.851463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505531-8cbc-4614-b9b3-ba31eac40d39 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:44.889049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:44.889664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:44.889946 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:44.890221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:44.959149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:44.960657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:45.031635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:45.065204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 558/1128] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:44 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:17:45.072780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:45.130087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:45.203600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765112ec-f57a-4547-b4f9-aeebbd370efb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:45.239289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118783-25d3-4603-8517-ac6d089214dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:45.239938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118783-25d3-4603-8517-ac6d089214dd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:45.240225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118783-25d3-4603-8517-ac6d089214dd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:45.240559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118783-25d3-4603-8517-ac6d089214dd None admin] PATH_INFO: `/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:45.241719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118783-25d3-4603-8517-ac6d089214dd None admin] RBAC: Authorizing `identity:get_project(project_id=d41d2a892a1445c28bdaa82a011d4ddd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:45.243163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118783-25d3-4603-8517-ac6d089214dd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:45.244944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 571/1129] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:45 2022] GET /identity/v3/projects/d41d2a892a1445c28bdaa82a011d4ddd => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:45.252623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:45.362770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a669a1-e364-4b63-8148-2065b8bd7625 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:45.397835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743e4e6-8da7-45eb-9db5-f7b9318c5e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:45.398470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e4e6-8da7-45eb-9db5-f7b9318c5e37 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:45.398745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e4e6-8da7-45eb-9db5-f7b9318c5e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:45.399022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e4e6-8da7-45eb-9db5-f7b9318c5e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:45.399818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743e4e6-8da7-45eb-9db5-f7b9318c5e37 None admin] RBAC: Adding query filter params (role.id=364a95d0748e48e9a538b4e6be293f91, scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd, user.id=3b4dca35d9bb4f8886d58d2b169551c5) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:45.400171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743e4e6-8da7-45eb-9db5-f7b9318c5e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:45.402245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743e4e6-8da7-45eb-9db5-f7b9318c5e37 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:45.408069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 559/1130] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:45 2022] GET /identity/v3/role_assignments?user.id=3b4dca35d9bb4f8886d58d2b169551c5&scope.project.id=d41d2a892a1445c28bdaa82a011d4ddd&role.id=364a95d0748e48e9a538b4e6be293f91 => generated 610 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:47.604769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a9247-ab5f-4b65-ba9c-002431e23836 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:47.605296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a9247-ab5f-4b65-ba9c-002431e23836 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:47.605296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a9247-ab5f-4b65-ba9c-002431e23836 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:47.606669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 572/1131] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:47.612304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1a89-bf32-4d29-9fab-564a2b0f0cc5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:47.612774 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1a89-bf32-4d29-9fab-564a2b0f0cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:47.612888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1a89-bf32-4d29-9fab-564a2b0f0cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:47.631742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7efc1a89-bf32-4d29-9fab-564a2b0f0cc5 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:47.632962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7efc1a89-bf32-4d29-9fab-564a2b0f0cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:47.662864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:47.702582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 560/1132] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:47 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:47.726182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a9247-ab5f-4b65-ba9c-002431e23836 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:47.828410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a9247-ab5f-4b65-ba9c-002431e23836 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:47.838319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:47.863443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4e420-a73c-4283-81b3-162692970b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:47.864084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4e420-a73c-4283-81b3-162692970b66 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:47.864364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4e420-a73c-4283-81b3-162692970b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:47.864662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4e420-a73c-4283-81b3-162692970b66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:47.865876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4e420-a73c-4283-81b3-162692970b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:47.867308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4e420-a73c-4283-81b3-162692970b66 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:47.872529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 573/1133] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:47 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:47.875410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1a89-bf32-4d29-9fab-564a2b0f0cc5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:47.975717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1a89-bf32-4d29-9fab-564a2b0f0cc5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:48.009915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2242c-ebbb-46a4-91ea-fca1e8af1dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:48.010520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2242c-ebbb-46a4-91ea-fca1e8af1dd4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:48.011048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2242c-ebbb-46a4-91ea-fca1e8af1dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:48.011048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2242c-ebbb-46a4-91ea-fca1e8af1dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:48.011679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2242c-ebbb-46a4-91ea-fca1e8af1dd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:48.013141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2242c-ebbb-46a4-91ea-fca1e8af1dd4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:48.127176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 561/1134] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:17:47 2022] POST /identity/v3/projects => generated 312 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:50.326365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738532f-b4f4-4c28-9346-a03b5ae60719 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:50.326896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738532f-b4f4-4c28-9346-a03b5ae60719 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:50.326896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738532f-b4f4-4c28-9346-a03b5ae60719 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:50.328280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 574/1135] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:50.333814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4f2c5-2b2b-4222-a3cf-f603b6f15805 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:50.334298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4f2c5-2b2b-4222-a3cf-f603b6f15805 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:50.334394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4f2c5-2b2b-4222-a3cf-f603b6f15805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:50.358573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e0d4f2c5-2b2b-4222-a3cf-f603b6f15805 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:50.360714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e0d4f2c5-2b2b-4222-a3cf-f603b6f15805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:50.377063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:50.447494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 562/1136] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:50 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:50.477554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738532f-b4f4-4c28-9346-a03b5ae60719 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:50.586502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738532f-b4f4-4c28-9346-a03b5ae60719 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:50.625064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e0abd-9c76-4812-af33-674e6f3926fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:50.625704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0abd-9c76-4812-af33-674e6f3926fb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:50.625988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0abd-9c76-4812-af33-674e6f3926fb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:50.626267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0abd-9c76-4812-af33-674e6f3926fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:50.627585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e0abd-9c76-4812-af33-674e6f3926fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:50.629080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e0abd-9c76-4812-af33-674e6f3926fb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:50.630718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 575/1137] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:17:50 2022] GET /identity/v3/domains/default => generated 200 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:50.637879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4f2c5-2b2b-4222-a3cf-f603b6f15805 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:50.737904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4f2c5-2b2b-4222-a3cf-f603b6f15805 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:50.772737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f4ec9-a3e1-473e-b4c8-1e4ab330240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:50.773475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4ec9-a3e1-473e-b4c8-1e4ab330240a None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:50.773763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4ec9-a3e1-473e-b4c8-1e4ab330240a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:50.774044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4ec9-a3e1-473e-b4c8-1e4ab330240a None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:50.775401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f4ec9-a3e1-473e-b4c8-1e4ab330240a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:50.776859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f4ec9-a3e1-473e-b4c8-1e4ab330240a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:50.903782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 563/1138] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:17:50 2022] POST /identity/v3/users => generated 287 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:53.040107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b6d0-3d1a-40fc-bd56-eb6f6cafca6a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.040667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b6d0-3d1a-40fc-bd56-eb6f6cafca6a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.040667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b6d0-3d1a-40fc-bd56-eb6f6cafca6a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.042017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 576/1139] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:53.048220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5073f-5c7e-4f73-803d-a10fdfc31ac7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.048723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5073f-5c7e-4f73-803d-a10fdfc31ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.048815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5073f-5c7e-4f73-803d-a10fdfc31ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.056094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:53.066672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2fc5073f-5c7e-4f73-803d-a10fdfc31ac7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:53.067861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2fc5073f-5c7e-4f73-803d-a10fdfc31ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:53.135446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 564/1140] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:53 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:53.159125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b6d0-3d1a-40fc-bd56-eb6f6cafca6a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.206311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:53.263359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b6d0-3d1a-40fc-bd56-eb6f6cafca6a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.302239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.302802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.303087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.303364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.313543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:53.318529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:53.319660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:53.320754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 577/1141] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:53.325913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5073f-5c7e-4f73-803d-a10fdfc31ac7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.425583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5073f-5c7e-4f73-803d-a10fdfc31ac7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.459894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.460548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.460901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.461158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.462666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:53.462779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.464326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.465858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:53.473395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 565/1142] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:53.480425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.580859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a65bf-12fb-40a6-aedf-440920767835 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.583057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:53.615063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.615891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.616397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.617133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] PATH_INFO: `/v3/users/af1ec9c55c1a46da91b07dee86ac2d63` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.630600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=af1ec9c55c1a46da91b07dee86ac2d63)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.632081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.634298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 578/1143] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/users/af1ec9c55c1a46da91b07dee86ac2d63 => generated 287 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:53.640353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.742234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37995eea-ce02-440b-a5b7-4e66cbbd2f91 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.777232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.777865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.778147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.778426 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.841134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:53.847478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.851997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.954883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 566/1144] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:17:53.963530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:54.088933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c00c54-5b16-4949-81a7-d454f2156fe1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:54.134273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c554295-eff7-4cd5-8db3-b587f75c7ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:54.134819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554295-eff7-4cd5-8db3-b587f75c7ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:54.135099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554295-eff7-4cd5-8db3-b587f75c7ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:54.135373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554295-eff7-4cd5-8db3-b587f75c7ca2 None admin] PATH_INFO: `/v3/projects/5b8f7d9431ba43eb88009931c8010992` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:54.136577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c554295-eff7-4cd5-8db3-b587f75c7ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8f7d9431ba43eb88009931c8010992)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:54.138021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c554295-eff7-4cd5-8db3-b587f75c7ca2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:54.139691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 579/1145] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/projects/5b8f7d9431ba43eb88009931c8010992 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:54.146045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:54.263101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:54.268920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563fb03-022c-4285-89ca-a6357806b3f1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:54.304092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f3b3b-7b1e-42bb-9fa7-23f504bc84e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:54.304747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f3b3b-7b1e-42bb-9fa7-23f504bc84e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:54.305033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f3b3b-7b1e-42bb-9fa7-23f504bc84e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:54.305303 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f3b3b-7b1e-42bb-9fa7-23f504bc84e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:54.307072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f3b3b-7b1e-42bb-9fa7-23f504bc84e5 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=5b8f7d9431ba43eb88009931c8010992, user.id=af1ec9c55c1a46da91b07dee86ac2d63) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:54.307402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f3b3b-7b1e-42bb-9fa7-23f504bc84e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:54.308869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f3b3b-7b1e-42bb-9fa7-23f504bc84e5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:54.315432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 567/1146] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:17:54 2022] GET /identity/v3/role_assignments?user.id=af1ec9c55c1a46da91b07dee86ac2d63&scope.project.id=5b8f7d9431ba43eb88009931c8010992&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:56.422962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7310e0-1fcd-4348-b14b-9752f8fe4890 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:56.423540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7310e0-1fcd-4348-b14b-9752f8fe4890 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:56.423540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7310e0-1fcd-4348-b14b-9752f8fe4890 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:56.424902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 580/1147] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:56.431184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34a955-f8d5-4ea9-9fc2-907d3696e7e9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:56.431670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34a955-f8d5-4ea9-9fc2-907d3696e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:56.431761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34a955-f8d5-4ea9-9fc2-907d3696e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:56.451906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ce34a955-f8d5-4ea9-9fc2-907d3696e7e9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:56.453143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ce34a955-f8d5-4ea9-9fc2-907d3696e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:56.523680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 568/1148] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:56 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:17:56.546257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7310e0-1fcd-4348-b14b-9752f8fe4890 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:56.577716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:56.650908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7310e0-1fcd-4348-b14b-9752f8fe4890 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:56.683547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:56.684115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:56.684399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:56.684706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:56.694522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:56.700013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:56.701189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:17:56.702045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 581/1149] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:17:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:56.707266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34a955-f8d5-4ea9-9fc2-907d3696e7e9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:56.801083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:56.811510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34a955-f8d5-4ea9-9fc2-907d3696e7e9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:56.846262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:56.846886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:56.847174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:56.847452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:56.848191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:56.848564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:56.849902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:56.858427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 569/1150] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:17:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:56.864574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:56.956793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:56.966268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1883b-fe08-4e52-92e3-9650798218e4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:57.001584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:57.002221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.002509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.002788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] PATH_INFO: `/v3/users/af1ec9c55c1a46da91b07dee86ac2d63` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.003965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] RBAC: Authorizing `identity:get_user(user_id=af1ec9c55c1a46da91b07dee86ac2d63)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:57.005433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:57.007050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 582/1151] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:56 2022] GET /identity/v3/users/af1ec9c55c1a46da91b07dee86ac2d63 => generated 287 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:57.014053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:57.145020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe5913-0e1c-4063-a890-a21c61125de9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:57.192483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37771dd5-2178-44ea-8e3e-395cbbd04613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:57.193134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37771dd5-2178-44ea-8e3e-395cbbd04613 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.193425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37771dd5-2178-44ea-8e3e-395cbbd04613 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.193704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37771dd5-2178-44ea-8e3e-395cbbd04613 None admin] PATH_INFO: `/v3/projects/5b8f7d9431ba43eb88009931c8010992` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.194863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37771dd5-2178-44ea-8e3e-395cbbd04613 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8f7d9431ba43eb88009931c8010992)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:57.196654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37771dd5-2178-44ea-8e3e-395cbbd04613 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:57.199110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 570/1152] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:17:57 2022] GET /identity/v3/projects/5b8f7d9431ba43eb88009931c8010992 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:57.205298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:57.311754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43c011-8dc3-4aac-95fa-260c7665bbec None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:57.346503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f3b19-6df2-4e49-b405-3ebfd798f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:57.347159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3b19-6df2-4e49-b405-3ebfd798f2e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.347438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3b19-6df2-4e49-b405-3ebfd798f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.347715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3b19-6df2-4e49-b405-3ebfd798f2e0 None admin] PATH_INFO: `/v3/projects/5b8f7d9431ba43eb88009931c8010992/users/af1ec9c55c1a46da91b07dee86ac2d63/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.349602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f3b19-6df2-4e49-b405-3ebfd798f2e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b8f7d9431ba43eb88009931c8010992, user_id=af1ec9c55c1a46da91b07dee86ac2d63, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:57.351028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f3b19-6df2-4e49-b405-3ebfd798f2e0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:57.434035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 583/1153] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:17:57 2022] PUT /identity/v3/projects/5b8f7d9431ba43eb88009931c8010992/users/af1ec9c55c1a46da91b07dee86ac2d63/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:59.485718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb308-b596-42b4-8dd0-1932598b180c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.486209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb308-b596-42b4-8dd0-1932598b180c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.486366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb308-b596-42b4-8dd0-1932598b180c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.487734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 571/1154] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:17:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:17:59.494026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb93688-8d48-4c76-9776-b7268b36d49b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.494472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb93688-8d48-4c76-9776-b7268b36d49b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.495094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb93688-8d48-4c76-9776-b7268b36d49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.513615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6fb93688-8d48-4c76-9776-b7268b36d49b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:59.514824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6fb93688-8d48-4c76-9776-b7268b36d49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:59.531038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:59.588608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 584/1155] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:17:59 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:17:59.613268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473eb308-b596-42b4-8dd0-1932598b180c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:59.641312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:59.751502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473eb308-b596-42b4-8dd0-1932598b180c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:59.784268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:59.784934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.785221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.785496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.800143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:17:59.805066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:17:59.806815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:17:59.807794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 572/1156] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:17:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:17:59.813307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb93688-8d48-4c76-9776-b7268b36d49b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:59.927260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb93688-8d48-4c76-9776-b7268b36d49b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:59.961930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:59.962553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.962839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.963116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.963837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:59.964150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:59.965656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:59.973992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 585/1157] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:17:59 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:59.981662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:00.064041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:00.082799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92789c-7542-4187-b717-c8dfac8249c8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:00.117714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:00.118945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:00.119230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:00.119503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] PATH_INFO: `/v3/users/af1ec9c55c1a46da91b07dee86ac2d63` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:00.120700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] RBAC: Authorizing `identity:get_user(user_id=af1ec9c55c1a46da91b07dee86ac2d63)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:00.122114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:00.123712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 573/1158] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:17:59 2022] GET /identity/v3/users/af1ec9c55c1a46da91b07dee86ac2d63 => generated 287 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:00.129569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:00.142305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:00.232000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca978cd-45dd-4c24-a4ba-94387e59fcea None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:00.266021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:00.266634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:00.266913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:00.267183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:00.335331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:00.337016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:00.348348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:00.437639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 586/1159] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:18:00 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:18:00.443568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:00.547245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75de1c-d7fc-473b-bc9e-1a8567dcd8c9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:00.580002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1accffa-6da0-4669-aaab-c7135c3c73b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:00.580679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1accffa-6da0-4669-aaab-c7135c3c73b2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:00.580967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1accffa-6da0-4669-aaab-c7135c3c73b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:00.581256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1accffa-6da0-4669-aaab-c7135c3c73b2 None admin] PATH_INFO: `/v3/projects/5b8f7d9431ba43eb88009931c8010992` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:00.582411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1accffa-6da0-4669-aaab-c7135c3c73b2 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8f7d9431ba43eb88009931c8010992)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:00.583846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1accffa-6da0-4669-aaab-c7135c3c73b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:00.585591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 574/1160] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:18:00 2022] GET /identity/v3/projects/5b8f7d9431ba43eb88009931c8010992 => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:18:00.592928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:00.707816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9359516-8df8-4b99-afea-c4838fe73ff0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:00.721783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:00.748831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa046-1a1d-4117-8717-98aa13225353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:00.749682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa046-1a1d-4117-8717-98aa13225353 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:00.749873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa046-1a1d-4117-8717-98aa13225353 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:00.750114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa046-1a1d-4117-8717-98aa13225353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:00.751081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baa046-1a1d-4117-8717-98aa13225353 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=5b8f7d9431ba43eb88009931c8010992, user.id=af1ec9c55c1a46da91b07dee86ac2d63) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:00.751397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baa046-1a1d-4117-8717-98aa13225353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:00.752894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baa046-1a1d-4117-8717-98aa13225353 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:00.760388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 587/1161] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:18:00 2022] GET /identity/v3/role_assignments?user.id=af1ec9c55c1a46da91b07dee86ac2d63&scope.project.id=5b8f7d9431ba43eb88009931c8010992&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:03.019723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778776-bf23-42b8-a645-b4715f7a17aa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:03.020308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778776-bf23-42b8-a645-b4715f7a17aa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:03.020308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778776-bf23-42b8-a645-b4715f7a17aa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:03.021665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 575/1162] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:18:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:18:03.027090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205aedf-4093-401a-b666-068d2175a2be None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:03.027511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205aedf-4093-401a-b666-068d2175a2be None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:03.027644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205aedf-4093-401a-b666-068d2175a2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:03.047801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7205aedf-4093-401a-b666-068d2175a2be None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:03.049039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7205aedf-4093-401a-b666-068d2175a2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:03.121918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 588/1163] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:18:03 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:18:03.145223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96778776-bf23-42b8-a645-b4715f7a17aa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:03.171398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:03.265646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96778776-bf23-42b8-a645-b4715f7a17aa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:03.306053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:03.306585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:03.306863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:03.307144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:03.377626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:03.379135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:03.397312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:03.490560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 576/1164] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:18:03 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:18:03.496766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205aedf-4093-401a-b666-068d2175a2be None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:03.598188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205aedf-4093-401a-b666-068d2175a2be None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:03.600596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:03.644833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780b1b8-1531-4dff-986d-94ff57ddb567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:03.645929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780b1b8-1531-4dff-986d-94ff57ddb567 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:03.646445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780b1b8-1531-4dff-986d-94ff57ddb567 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:03.646948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780b1b8-1531-4dff-986d-94ff57ddb567 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:03.658260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780b1b8-1531-4dff-986d-94ff57ddb567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:03.659694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780b1b8-1531-4dff-986d-94ff57ddb567 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7780b1b8-1531-4dff-986d-94ff57ddb567 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:18:03.677377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:18:03.678731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:18:03.680043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 589/1165] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:18:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:18:03.684954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:03.804428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556846f-e57b-434d-9073-4079411c6583 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:03.819954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:03.838172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eaacf-798c-4fe4-8470-410855ee9639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:03.838782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eaacf-798c-4fe4-8470-410855ee9639 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:03.839064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eaacf-798c-4fe4-8470-410855ee9639 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:03.839340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eaacf-798c-4fe4-8470-410855ee9639 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:03.840034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eaacf-798c-4fe4-8470-410855ee9639 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:03.840342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eaacf-798c-4fe4-8470-410855ee9639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:03.841733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eaacf-798c-4fe4-8470-410855ee9639 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:03.848806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 577/1166] 10.222.0.128 () {56 vars in 1120 bytes} [Tue Jul 26 15:18:03 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:05.871744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495753b-1685-49f6-ad47-29c8b24c75ff None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:05.872261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495753b-1685-49f6-ad47-29c8b24c75ff None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:05.872261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495753b-1685-49f6-ad47-29c8b24c75ff None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:05.873697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 590/1167] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:18:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:18:05.878767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8c20-adad-4f8f-8251-20a99ef012cd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:05.879204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8c20-adad-4f8f-8251-20a99ef012cd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:05.879333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8c20-adad-4f8f-8251-20a99ef012cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:05.897838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6d3d8c20-adad-4f8f-8251-20a99ef012cd None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:05.899846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6d3d8c20-adad-4f8f-8251-20a99ef012cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:05.965067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 578/1168] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:18:05 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:18:05.987803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495753b-1685-49f6-ad47-29c8b24c75ff None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:06.087447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495753b-1685-49f6-ad47-29c8b24c75ff None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:06.120516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29258834-beeb-429f-95bc-1c8734796580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:06.121120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29258834-beeb-429f-95bc-1c8734796580 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:06.121398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29258834-beeb-429f-95bc-1c8734796580 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:06.121668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29258834-beeb-429f-95bc-1c8734796580 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:06.122416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29258834-beeb-429f-95bc-1c8734796580 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:06.123734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29258834-beeb-429f-95bc-1c8734796580 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:06.199710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 591/1169] 10.222.0.128 () {60 vars in 1143 bytes} [Tue Jul 26 15:18:05 2022] POST /identity/v3/domains => generated 268 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:08.252225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9d738-88b4-4ecb-a4d8-77e8a28347fa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:08.252813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9d738-88b4-4ecb-a4d8-77e8a28347fa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:08.252813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9d738-88b4-4ecb-a4d8-77e8a28347fa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:08.254128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 579/1170] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:18:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:18:08.259451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d119b-24e7-42a5-a1e9-fdb4f5091e35 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:08.259884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d119b-24e7-42a5-a1e9-fdb4f5091e35 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:08.260034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d119b-24e7-42a5-a1e9-fdb4f5091e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:08.277902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-185d119b-24e7-42a5-a1e9-fdb4f5091e35 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:08.279074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-185d119b-24e7-42a5-a1e9-fdb4f5091e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:08.296283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:08.347076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 592/1171] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:18:08 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:18:08.370887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9d738-88b4-4ecb-a4d8-77e8a28347fa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:08.470944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9d738-88b4-4ecb-a4d8-77e8a28347fa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:08.482616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:08.505539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34193fdd-1603-49ca-aaac-e9a4daf28540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:08.506159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193fdd-1603-49ca-aaac-e9a4daf28540 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:08.506443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193fdd-1603-49ca-aaac-e9a4daf28540 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:08.506719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193fdd-1603-49ca-aaac-e9a4daf28540 None admin] PATH_INFO: `/v3/domains/ce129d9b6deb42c783387645c20b26d2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:08.517854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34193fdd-1603-49ca-aaac-e9a4daf28540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce129d9b6deb42c783387645c20b26d2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:08.519289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34193fdd-1603-49ca-aaac-e9a4daf28540 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:08.520915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 580/1172] 10.222.0.128 () {56 vars in 1188 bytes} [Tue Jul 26 15:18:08 2022] GET /identity/v3/domains/ce129d9b6deb42c783387645c20b26d2 => generated 268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:08.531631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d119b-24e7-42a5-a1e9-fdb4f5091e35 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:08.628378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d119b-24e7-42a5-a1e9-fdb4f5091e35 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:08.661382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63f60-a473-451c-87cd-56647d57c770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:08.662006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63f60-a473-451c-87cd-56647d57c770 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:08.662292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63f60-a473-451c-87cd-56647d57c770 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:08.662572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63f60-a473-451c-87cd-56647d57c770 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:08.663265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63f60-a473-451c-87cd-56647d57c770 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:08.665463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63f60-a473-451c-87cd-56647d57c770 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:08.804497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 593/1173] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:18:08 2022] POST /identity/v3/projects => generated 362 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:10.884258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b998a-d735-4fca-ac20-be1c4cb93009 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:10.884807 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b998a-d735-4fca-ac20-be1c4cb93009 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:10.884807 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b998a-d735-4fca-ac20-be1c4cb93009 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:10.886112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 581/1174] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:18:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:18:10.891823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25b7e-e5cc-4f21-ae11-e2fb638573ce None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:10.892335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25b7e-e5cc-4f21-ae11-e2fb638573ce None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:10.892437 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25b7e-e5cc-4f21-ae11-e2fb638573ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:10.905801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:10.911871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-80f25b7e-e5cc-4f21-ae11-e2fb638573ce None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:10.913084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-80f25b7e-e5cc-4f21-ae11-e2fb638573ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:10.980960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 594/1175] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:18:10 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:18:11.005576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b998a-d735-4fca-ac20-be1c4cb93009 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:11.109889 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b998a-d735-4fca-ac20-be1c4cb93009 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:11.142765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878fc20-28e1-4e85-8770-eb1971b125ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:11.143339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878fc20-28e1-4e85-8770-eb1971b125ed None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:11.143626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878fc20-28e1-4e85-8770-eb1971b125ed None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:11.143905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878fc20-28e1-4e85-8770-eb1971b125ed None admin] PATH_INFO: `/v3/domains/ce129d9b6deb42c783387645c20b26d2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:11.145115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878fc20-28e1-4e85-8770-eb1971b125ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce129d9b6deb42c783387645c20b26d2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:11.146551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878fc20-28e1-4e85-8770-eb1971b125ed None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:11.148316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 582/1176] 10.222.0.128 () {56 vars in 1188 bytes} [Tue Jul 26 15:18:11 2022] GET /identity/v3/domains/ce129d9b6deb42c783387645c20b26d2 => generated 268 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:11.154662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f25b7e-e5cc-4f21-ae11-e2fb638573ce None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:11.253514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f25b7e-e5cc-4f21-ae11-e2fb638573ce None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:11.294751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ccb66-df18-4253-970a-d87123fcab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:11.295389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ccb66-df18-4253-970a-d87123fcab7f None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:11.295678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ccb66-df18-4253-970a-d87123fcab7f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:11.295957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ccb66-df18-4253-970a-d87123fcab7f None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:11.296684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ccb66-df18-4253-970a-d87123fcab7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:11.298134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ccb66-df18-4253-970a-d87123fcab7f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:11.307193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:11.397617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 595/1177] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:18:11 2022] POST /identity/v3/users => generated 312 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:13.478405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae672ada-80ba-43c1-9d2a-f35721e2540c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.478941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae672ada-80ba-43c1-9d2a-f35721e2540c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.478941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae672ada-80ba-43c1-9d2a-f35721e2540c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.480288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 583/1178] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:18:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:18:13.485757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f486-906b-46fd-b0c5-309b6880b394 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.486131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f486-906b-46fd-b0c5-309b6880b394 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.486309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f486-906b-46fd-b0c5-309b6880b394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.504633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5c64f486-906b-46fd-b0c5-309b6880b394 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:13.505807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5c64f486-906b-46fd-b0c5-309b6880b394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:13.573665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 596/1179] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:18:13 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:18:13.598862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae672ada-80ba-43c1-9d2a-f35721e2540c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.641541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:13.706203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae672ada-80ba-43c1-9d2a-f35721e2540c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.740457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.741050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.741337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.742242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.753202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:18:13.758674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:18:13.759798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:18:13.760899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 584/1180] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:18:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:18:13.766338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64f486-906b-46fd-b0c5-309b6880b394 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.868001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64f486-906b-46fd-b0c5-309b6880b394 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.894376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:13.906688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.907332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.907614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.907887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.908703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:13.909012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:13.910390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.919687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 597/1181] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:18:13 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:13.925827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.027822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:14.031597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977b96-0feb-43c8-bf59-c8da365493f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.068380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.069086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.069369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.070339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] PATH_INFO: `/v3/users/9e89f722df634b0aa0539613cb98f90d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.083697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] RBAC: Authorizing `identity:get_user(user_id=9e89f722df634b0aa0539613cb98f90d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.085167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.086901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 585/1182] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:18:13 2022] GET /identity/v3/users/9e89f722df634b0aa0539613cb98f90d => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:14.092880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.192672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffdba7-c359-4538-a88b-58ad8ddc348b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.226435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.227122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.227408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.227686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.280418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:14.296090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.297570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.396235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 598/1183] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:18:14 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:18:14.403822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.504180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d243de-20b4-4653-82c3-72275ff378e1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.537598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aceb06e-c70e-4f6b-a8a5-db5af756a568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.538155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceb06e-c70e-4f6b-a8a5-db5af756a568 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.539013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceb06e-c70e-4f6b-a8a5-db5af756a568 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.539290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceb06e-c70e-4f6b-a8a5-db5af756a568 None admin] PATH_INFO: `/v3/projects/22dcf5b6d07740d9863b1b307ab600ca` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.540446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aceb06e-c70e-4f6b-a8a5-db5af756a568 None admin] RBAC: Authorizing `identity:get_project(project_id=22dcf5b6d07740d9863b1b307ab600ca)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.541895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aceb06e-c70e-4f6b-a8a5-db5af756a568 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.543529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 586/1184] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:18:14 2022] GET /identity/v3/projects/22dcf5b6d07740d9863b1b307ab600ca => generated 362 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:14.549706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.633419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:14.649451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f696c6a-7f48-4fb3-832e-25289a6f7109 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.684251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a2380-f7dd-415c-8595-9aaefbfb9e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.684901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a2380-f7dd-415c-8595-9aaefbfb9e9a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.685189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a2380-f7dd-415c-8595-9aaefbfb9e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.685471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a2380-f7dd-415c-8595-9aaefbfb9e9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.686261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a2380-f7dd-415c-8595-9aaefbfb9e9a None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=22dcf5b6d07740d9863b1b307ab600ca, user.id=9e89f722df634b0aa0539613cb98f90d) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:14.686580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a2380-f7dd-415c-8595-9aaefbfb9e9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.688017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a2380-f7dd-415c-8595-9aaefbfb9e9a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.693349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 599/1185] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:18:14 2022] GET /identity/v3/role_assignments?user.id=9e89f722df634b0aa0539613cb98f90d&scope.project.id=22dcf5b6d07740d9863b1b307ab600ca&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:16.664611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45b8a6-acf9-4a1f-b727-9cde39bfab82 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:16.665134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45b8a6-acf9-4a1f-b727-9cde39bfab82 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:16.665243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45b8a6-acf9-4a1f-b727-9cde39bfab82 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:16.666640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 587/1186] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:18:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:18:16.672030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde9e7-c5fb-49e2-a3bf-7f9d2ab52d68 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:16.672456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde9e7-c5fb-49e2-a3bf-7f9d2ab52d68 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:16.672633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde9e7-c5fb-49e2-a3bf-7f9d2ab52d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:16.690229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-94bde9e7-c5fb-49e2-a3bf-7f9d2ab52d68 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:16.691408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-94bde9e7-c5fb-49e2-a3bf-7f9d2ab52d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:16.758372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 600/1187] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:18:16 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:18:16.781032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45b8a6-acf9-4a1f-b727-9cde39bfab82 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:16.806230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:16.881455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45b8a6-acf9-4a1f-b727-9cde39bfab82 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:16.915618 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:16.916260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:16.916668 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:16.916938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:16.926936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:18:16.931859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:18:16.933035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:18:16.934142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 588/1188] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:18:16.939201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bde9e7-c5fb-49e2-a3bf-7f9d2ab52d68 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:17.015919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:17.036992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bde9e7-c5fb-49e2-a3bf-7f9d2ab52d68 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:17.070946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:17.071556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:17.071837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:17.072113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:17.072869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:17.073168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:17.074535 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:17.082478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 601/1189] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:18:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:17.089265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:17.172250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:17.187825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55fa73-2b47-46d8-b881-b7f7cb58e2d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:17.222459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:17.223089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:17.223373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:17.223647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] PATH_INFO: `/v3/users/9e89f722df634b0aa0539613cb98f90d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:17.224835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=9e89f722df634b0aa0539613cb98f90d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:17.226766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:17.228350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 589/1190] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:18:17 2022] GET /identity/v3/users/9e89f722df634b0aa0539613cb98f90d => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:17.234490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:17.351433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3fcd5-e54e-4996-8b3d-2aa2ff07d258 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:17.386230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062667-c6a4-4d85-b554-2404a4f1238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:17.386866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062667-c6a4-4d85-b554-2404a4f1238e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:17.387152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062667-c6a4-4d85-b554-2404a4f1238e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:17.387433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062667-c6a4-4d85-b554-2404a4f1238e None admin] PATH_INFO: `/v3/projects/22dcf5b6d07740d9863b1b307ab600ca` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:17.388657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062667-c6a4-4d85-b554-2404a4f1238e None admin] RBAC: Authorizing `identity:get_project(project_id=22dcf5b6d07740d9863b1b307ab600ca)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:17.390087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062667-c6a4-4d85-b554-2404a4f1238e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:17.391753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 602/1191] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:18:17 2022] GET /identity/v3/projects/22dcf5b6d07740d9863b1b307ab600ca => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:17.398280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:17.497004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3bc5c-0ddc-497f-bc37-06d434914ad2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:17.530607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f7137-d6ad-4f45-8736-d306a9003426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:17.531264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7137-d6ad-4f45-8736-d306a9003426 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:17.531547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7137-d6ad-4f45-8736-d306a9003426 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:17.531821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7137-d6ad-4f45-8736-d306a9003426 None admin] PATH_INFO: `/v3/projects/22dcf5b6d07740d9863b1b307ab600ca/users/9e89f722df634b0aa0539613cb98f90d/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:17.533759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f7137-d6ad-4f45-8736-d306a9003426 None admin] RBAC: Authorizing `identity:create_grant(project_id=22dcf5b6d07740d9863b1b307ab600ca, user_id=9e89f722df634b0aa0539613cb98f90d, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:17.535181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f7137-d6ad-4f45-8736-d306a9003426 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:17.620707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 590/1192] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:18:17 2022] PUT /identity/v3/projects/22dcf5b6d07740d9863b1b307ab600ca/users/9e89f722df634b0aa0539613cb98f90d/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:19.595283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317a22-932e-461a-9c7a-699cd1211c13 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:19.595797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317a22-932e-461a-9c7a-699cd1211c13 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:19.595912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317a22-932e-461a-9c7a-699cd1211c13 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:19.597237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 603/1193] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:18:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:18:19.603407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cc811-bd6d-4176-97dc-2dbc8aec2de3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:19.603835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cc811-bd6d-4176-97dc-2dbc8aec2de3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:19.603965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cc811-bd6d-4176-97dc-2dbc8aec2de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:19.622441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fc9cc811-bd6d-4176-97dc-2dbc8aec2de3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:19.623621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc9cc811-bd6d-4176-97dc-2dbc8aec2de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:19.630790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:19.688723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 591/1194] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:18:19 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:18:19.712441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7317a22-932e-461a-9c7a-699cd1211c13 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:19.723492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:19.812531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7317a22-932e-461a-9c7a-699cd1211c13 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:19.844803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:19.845434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:19.845714 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:19.845989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:19.857169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:18:19.862256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:18:19.863389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:18:19.864276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 604/1195] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:18:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:18:19.870165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cc811-bd6d-4176-97dc-2dbc8aec2de3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:19.966025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cc811-bd6d-4176-97dc-2dbc8aec2de3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:19.999757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:20.000373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:20.000687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:20.000972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:20.001688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:20.001998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:20.003394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:20.011911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 592/1196] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:18:19 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:20.017741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:20.082841 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:20.119404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb2b3-fc6d-4b36-aa15-e31e7006fcfc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:20.152427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:20.153277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:20.153578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:20.153834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] PATH_INFO: `/v3/users/9e89f722df634b0aa0539613cb98f90d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:20.154991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] RBAC: Authorizing `identity:get_user(user_id=9e89f722df634b0aa0539613cb98f90d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:20.156411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:20.158032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 605/1197] 10.222.0.128 () {56 vars in 1182 bytes} [Tue Jul 26 15:18:20 2022] GET /identity/v3/users/9e89f722df634b0aa0539613cb98f90d => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:20.164575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:20.168297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:20.265611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aef72-8586-4d75-bca3-9804d2ded70d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:20.299516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:20.300139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:20.300427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:20.300732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:20.369123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:20.370580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:20.373265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:20.473477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 593/1198] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:18:20 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:18:20.479959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:20.579547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7354d09-f519-4d6f-b8a8-f2b769f8096f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:20.614771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387c2f9-ebb4-43cb-8483-8589cc6311a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:20.615434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387c2f9-ebb4-43cb-8483-8589cc6311a5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:20.615679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387c2f9-ebb4-43cb-8483-8589cc6311a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:20.615991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387c2f9-ebb4-43cb-8483-8589cc6311a5 None admin] PATH_INFO: `/v3/projects/22dcf5b6d07740d9863b1b307ab600ca` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:20.617200 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387c2f9-ebb4-43cb-8483-8589cc6311a5 None admin] RBAC: Authorizing `identity:get_project(project_id=22dcf5b6d07740d9863b1b307ab600ca)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:20.618632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387c2f9-ebb4-43cb-8483-8589cc6311a5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:20.620303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 606/1199] 10.222.0.128 () {56 vars in 1191 bytes} [Tue Jul 26 15:18:20 2022] GET /identity/v3/projects/22dcf5b6d07740d9863b1b307ab600ca => generated 362 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:20.626467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:20.723030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129823-2d97-42cd-80dc-46b2bd853cb7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:20.726101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:20.757961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db254e59-db89-49bf-a79f-5495f9a1c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:20.758632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db254e59-db89-49bf-a79f-5495f9a1c284 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:20.758920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db254e59-db89-49bf-a79f-5495f9a1c284 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:20.759239 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db254e59-db89-49bf-a79f-5495f9a1c284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:20.760124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db254e59-db89-49bf-a79f-5495f9a1c284 None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=22dcf5b6d07740d9863b1b307ab600ca, user.id=9e89f722df634b0aa0539613cb98f90d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:20.760489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db254e59-db89-49bf-a79f-5495f9a1c284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:20.761966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db254e59-db89-49bf-a79f-5495f9a1c284 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:20.768505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 594/1200] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:18:20 2022] GET /identity/v3/role_assignments?user.id=9e89f722df634b0aa0539613cb98f90d&scope.project.id=22dcf5b6d07740d9863b1b307ab600ca&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:30.610460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:30.767057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=16 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:34.062656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5c720-9cb1-4e06-82b3-b5a3ae0e9e55 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:34.063229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5c720-9cb1-4e06-82b3-b5a3ae0e9e55 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:34.063229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5c720-9cb1-4e06-82b3-b5a3ae0e9e55 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:34.065236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 607/1201] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:19:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:19:34.070588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb190-6ae4-49ae-bf4e-8bf7d4f7d935 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:34.071004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb190-6ae4-49ae-bf4e-8bf7d4f7d935 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:34.071152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb190-6ae4-49ae-bf4e-8bf7d4f7d935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:34.090025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3c0fb190-6ae4-49ae-bf4e-8bf7d4f7d935 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:34.091329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3c0fb190-6ae4-49ae-bf4e-8bf7d4f7d935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:34.161607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 595/1202] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:19:34 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:19:34.185635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5c720-9cb1-4e06-82b3-b5a3ae0e9e55 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:34.286665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5c720-9cb1-4e06-82b3-b5a3ae0e9e55 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:34.320740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:34.321429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:34.321715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:34.321995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:34.333472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:34.334953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:34.349799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:34.350844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:34.351866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:34.351866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:34.351866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:34.351866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:34.351866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 608/1203] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:34.357414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fb190-6ae4-49ae-bf4e-8bf7d4f7d935 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:34.457078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fb190-6ae4-49ae-bf4e-8bf7d4f7d935 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:34.486605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:34.492229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae925e-9923-40e9-bb08-b453b8247d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:34.492924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae925e-9923-40e9-bb08-b453b8247d40 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:34.493206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae925e-9923-40e9-bb08-b453b8247d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:34.493482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae925e-9923-40e9-bb08-b453b8247d40 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:34.494176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae925e-9923-40e9-bb08-b453b8247d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:34.494481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae925e-9923-40e9-bb08-b453b8247d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:34.495844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae925e-9923-40e9-bb08-b453b8247d40 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:34.506932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 596/1204] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:34.513043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:34.566518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:34.614107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3bdd0-1367-461a-9f8f-9cd0ec930b86 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:34.648791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1e91-8ab9-461e-b50f-124c7f23b5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:34.649415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1e91-8ab9-461e-b50f-124c7f23b5ca None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:34.649697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1e91-8ab9-461e-b50f-124c7f23b5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:34.649973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1e91-8ab9-461e-b50f-124c7f23b5ca None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:34.650652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1e91-8ab9-461e-b50f-124c7f23b5ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:34.652089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1e91-8ab9-461e-b50f-124c7f23b5ca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:34.745300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 609/1205] 10.222.0.128 () {60 vars in 1137 bytes} [Tue Jul 26 15:19:34 2022] POST /identity/v3/users => generated 252 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:19:36.786449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da9f99-afe3-42ff-be0d-a5f798a73b13 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:36.786966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da9f99-afe3-42ff-be0d-a5f798a73b13 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:36.786966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da9f99-afe3-42ff-be0d-a5f798a73b13 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:36.788276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 597/1206] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:19:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:19:36.793504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882097b-ac33-4735-bb53-32e4a192419f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:36.793946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882097b-ac33-4735-bb53-32e4a192419f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:36.794070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882097b-ac33-4735-bb53-32e4a192419f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:36.812810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8882097b-ac33-4735-bb53-32e4a192419f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:36.814012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8882097b-ac33-4735-bb53-32e4a192419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:36.882547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 610/1207] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:19:36 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:19:36.905663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da9f99-afe3-42ff-be0d-a5f798a73b13 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:37.010312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da9f99-afe3-42ff-be0d-a5f798a73b13 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:37.043811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:37.044377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:37.044687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:37.044961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:37.055509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:19:37.060385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:37.061563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:37.062480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 598/1208] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:19:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:37.069015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882097b-ac33-4735-bb53-32e4a192419f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:37.109156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:37.167698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882097b-ac33-4735-bb53-32e4a192419f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:37.202704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:37.203334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:37.203622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:37.203901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:37.204659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:37.205621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:37.206993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:37.215176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 611/1209] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:19:37 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:37.221547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:37.273603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:37.327262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59800135-d9af-4cb7-b876-ae07b9c1cb94 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:37.360583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:37.361705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:37.362056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:37.362361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:37.373177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:37.380256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:37.389217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:37.390088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:37.391131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:37.391131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:37.391131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:37.391131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:37.391131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 599/1210] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:37.397137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:37.496331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:37.499298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bdc95-3cbe-4d79-b87e-80327687ef3d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:37.532087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:37.532727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:37.533684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:37.533973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:37.534665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:37.534968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:37.536329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:37.547750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 612/1211] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:37.553908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:37.660075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e202d-c820-4286-80a8-7a46236455b3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:37.661556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:37.694949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:37.695591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:37.695880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:37.696159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:37.703701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:37.706705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:19:37.717274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:37.718449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:37.719653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 600/1212] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:37.724560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:37.825623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8136-a3a9-44b5-afb3-0608838c2f80 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:37.860537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:37.861620 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:37.861953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:37.862236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:37.863033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:37.863340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:37.864811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:37.883682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 613/1213] 10.222.0.128 () {56 vars in 1148 bytes} [Tue Jul 26 15:19:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:37.889691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:37.993330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc5d90-ddb4-4572-b7f8-a4de4fca8133 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:38.026963 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:38.027622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:38.027915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:38.028196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:38.058257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:38.096071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:38.097692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:38.198811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 601/1214] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:19:37 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:19:38.205389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:38.225192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:38.305840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b76292-d907-4eab-84fe-6f3e41922a7a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:38.341746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:38.342389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:38.342672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:38.342946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:38.354816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:38.360746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:38.375898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:38.377856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:38.379223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:38.379223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:38.379223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:38.379223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:38.379223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 614/1215] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:38.384695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:38.446614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:38.486614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f3599-4085-45dc-bc53-6d23d29db28f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:38.522161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:38.522776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:38.523061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:38.523340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:38.524028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:38.524405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:38.525834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:38.537126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 602/1216] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:38.542945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:38.613373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:38.646263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ea744-60ba-49c4-b5e7-af489b63f0e2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:38.679570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:38.680206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:38.680581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:38.680865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:38.687388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:38.688966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:19:38.700988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:38.702096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:38.703050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 615/1217] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:19:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:38.709193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:38.810927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad339-4b1d-4b93-a0f3-8e686f1038af None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:38.834164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:38.845460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e9d87-983e-4c0d-8c58-ca7b8a454f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:38.846125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9d87-983e-4c0d-8c58-ca7b8a454f57 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:38.846406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9d87-983e-4c0d-8c58-ca7b8a454f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:38.846684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9d87-983e-4c0d-8c58-ca7b8a454f57 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:38.847439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e9d87-983e-4c0d-8c58-ca7b8a454f57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:38.847748 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e9d87-983e-4c0d-8c58-ca7b8a454f57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:38.849211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e9d87-983e-4c0d-8c58-ca7b8a454f57 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:38.861783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 603/1218] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:19:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:38.868176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:38.973335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303074d-ddae-4c32-8ab3-57bb52b5f758 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:39.002947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:39.009183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf0937-1a5e-4659-a6d1-4ebcf7a42d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:39.009811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf0937-1a5e-4659-a6d1-4ebcf7a42d8c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:39.010099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf0937-1a5e-4659-a6d1-4ebcf7a42d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:39.010378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf0937-1a5e-4659-a6d1-4ebcf7a42d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:39.011171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf0937-1a5e-4659-a6d1-4ebcf7a42d8c None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=16e8966ae76242898bc996cd039dbe31) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:39.011537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf0937-1a5e-4659-a6d1-4ebcf7a42d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:39.013047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf0937-1a5e-4659-a6d1-4ebcf7a42d8c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:39.018493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 616/1219] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:19:38 2022] GET /identity/v3/role_assignments?user.id=16e8966ae76242898bc996cd039dbe31&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 260 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:41.023796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73753-079b-4d1d-b217-7f3227bbedde None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:41.024323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73753-079b-4d1d-b217-7f3227bbedde None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:41.024437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73753-079b-4d1d-b217-7f3227bbedde None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:41.025741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 604/1220] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:19:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:19:41.032523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32611f7a-a6f4-4468-b952-d3df84f057f6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:41.032523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32611f7a-a6f4-4468-b952-d3df84f057f6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:41.032523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32611f7a-a6f4-4468-b952-d3df84f057f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:41.049837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-32611f7a-a6f4-4468-b952-d3df84f057f6 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:41.051026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-32611f7a-a6f4-4468-b952-d3df84f057f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:41.117789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 617/1221] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:19:41 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:19:41.141135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed73753-079b-4d1d-b217-7f3227bbedde None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:41.243908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed73753-079b-4d1d-b217-7f3227bbedde None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:41.277209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:41.277780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:41.278059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:41.278332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:41.288238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:19:41.293076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:41.299083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:41.300768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 605/1222] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:19:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:41.301677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32611f7a-a6f4-4468-b952-d3df84f057f6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:41.400958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32611f7a-a6f4-4468-b952-d3df84f057f6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:41.425184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:41.435265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:41.435878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:41.436163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:41.436436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:41.437183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:41.437486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:41.438847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:41.446966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 618/1223] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:19:41 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:41.452782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:41.497392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:41.552915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c33e5-d675-40ca-a94b-a014375d16d4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:41.587085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:41.587754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:41.588043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:41.588323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:41.599337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:41.600973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:41.617019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:41.623359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:41.624264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:41.624264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:41.624264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:41.624264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:41.624264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 606/1224] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:41.624572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:41.725005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0794517-b860-4496-861a-3bf34d8c3ed2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:41.758238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:41.758848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:41.759128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:41.759402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:41.760085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:41.760385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:41.761774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:41.773903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 619/1225] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:41.779708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:41.874794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:41.881447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7538a61-a5bb-4571-bcfb-00dbdfd25dd9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:41.915902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:41.916580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:41.916872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:41.917157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:41.924842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:41.926365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:19:41.937372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:41.939151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:41.940961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 607/1226] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:41.945009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:41.995872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:42.049339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792b6ac-32ba-4249-982b-f35dcb45b317 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:42.083057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:42.083674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:42.083958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:42.084233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:42.085068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:42.085379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:42.086804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:42.104739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 620/1227] 10.222.0.128 () {56 vars in 1148 bytes} [Tue Jul 26 15:19:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:42.110386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:42.212998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4435e4d-2646-4a90-a35c-8e493108e074 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:42.247357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:42.247988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:42.248270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:42.248570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:42.259540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:42.261042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:42.275990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:42.276990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:42.277902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:42.277902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:42.277902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:42.277902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:42.277902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 608/1228] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:19:42.283048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:42.386276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321692d0-eb18-4ba2-95e2-97d580a9587e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:42.388688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:42.423000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:42.424359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:42.424701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:42.425076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:42.425813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:42.426121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:42.427548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:42.439610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 621/1229] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:42.445660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:42.455985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:42.548972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2926c-6495-4ab7-b34f-c2e80d363ed2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:42.582088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:42.582754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:42.583037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:42.583313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:42.590041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:42.591503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:19:42.603070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:42.605415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:42.606546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 609/1230] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:19:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:42.610129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:42.710245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23f7a0-1d8d-4d27-9111-b51e94a6819a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:42.743938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102edbf-3df6-4abe-90fa-353e195c4b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:42.744587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102edbf-3df6-4abe-90fa-353e195c4b61 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:42.744871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102edbf-3df6-4abe-90fa-353e195c4b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:42.745146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102edbf-3df6-4abe-90fa-353e195c4b61 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:42.745894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102edbf-3df6-4abe-90fa-353e195c4b61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:42.746201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102edbf-3df6-4abe-90fa-353e195c4b61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:42.747631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102edbf-3df6-4abe-90fa-353e195c4b61 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:42.760488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 622/1231] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:42.766473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:42.832322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:42.867591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ab06b-63b8-4d2b-9977-43996109d80b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:42.903582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181d64b-1a3f-4fce-8f9e-304ce208ced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:42.904242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d64b-1a3f-4fce-8f9e-304ce208ced6 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:42.904573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d64b-1a3f-4fce-8f9e-304ce208ced6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:42.904856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d64b-1a3f-4fce-8f9e-304ce208ced6 None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/16e8966ae76242898bc996cd039dbe31/roles/224e0c1f1992436bbcf1ba18913cb81b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:42.920155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181d64b-1a3f-4fce-8f9e-304ce208ced6 None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=16e8966ae76242898bc996cd039dbe31, role_id=224e0c1f1992436bbcf1ba18913cb81b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:42.921737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181d64b-1a3f-4fce-8f9e-304ce208ced6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:43.035593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 610/1232] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:19:42 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/16e8966ae76242898bc996cd039dbe31/roles/224e0c1f1992436bbcf1ba18913cb81b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:19:45.062144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54298c10-2de2-4e8a-818f-a8177bf8070a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:45.062700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54298c10-2de2-4e8a-818f-a8177bf8070a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:45.062700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54298c10-2de2-4e8a-818f-a8177bf8070a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:45.065048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 623/1233] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:19:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:19:45.070315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e500db9-ba0d-4337-833f-66dfc1ac7a9a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:45.070732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e500db9-ba0d-4337-833f-66dfc1ac7a9a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:45.070878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e500db9-ba0d-4337-833f-66dfc1ac7a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:45.088760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1e500db9-ba0d-4337-833f-66dfc1ac7a9a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:45.089969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1e500db9-ba0d-4337-833f-66dfc1ac7a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:45.158375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 611/1234] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:19:45 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:19:45.181962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54298c10-2de2-4e8a-818f-a8177bf8070a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:45.206785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:45.284171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54298c10-2de2-4e8a-818f-a8177bf8070a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:45.318998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:45.319568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:45.319852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:45.320127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:45.330562 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:19:45.335453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:45.346255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:45.347184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 624/1235] 10.222.0.128 () {56 vars in 1107 bytes} [Tue Jul 26 15:19:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:45.347886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e500db9-ba0d-4337-833f-66dfc1ac7a9a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:45.389921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:45.442584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e500db9-ba0d-4337-833f-66dfc1ac7a9a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:45.480133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:45.480775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:45.481060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:45.481333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:45.482047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:45.482350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:45.483711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:45.492814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 612/1236] 10.222.0.128 () {56 vars in 1108 bytes} [Tue Jul 26 15:19:45 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:45.498588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:45.585174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:45.601326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b915d-b22e-4ab6-9a21-f939e74b6090 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:45.636162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:45.636818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:45.637099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:45.637372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:45.647217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:45.648678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:45.663333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:45.664379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:45.666167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:45.666167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:45.666167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:45.666167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:45.666167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 625/1237] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:45.670527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:45.775836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b3012-20b8-4361-a9c2-679b9cc3158a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:45.786906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:45.813258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:45.814571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:45.814930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:45.815216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:45.815920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:45.816235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:45.817744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:45.829836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 613/1238] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:45.838888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:45.941861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5036e0-8b8d-43f8-b827-afc8b0f21f90 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:45.976669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:45.977728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:45.978013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:45.978288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:45.981020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:45.986385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:45.988090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:19:45.999118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:46.000286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:46.002377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 626/1239] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:46.006796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:46.108225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ef34-baca-43f5-a81a-190b88b3175d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:46.144862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:46.145477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:46.145758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:46.146032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:46.146831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:46.147139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:46.148720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:46.166013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 614/1240] 10.222.0.128 () {56 vars in 1148 bytes} [Tue Jul 26 15:19:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:46.171780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:46.275581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc8d67-02b1-4363-8bf2-057a02fc3460 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:46.309252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:46.309864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:46.310147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:46.310423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:46.371914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:46.378668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:46.380139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:46.480851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 627/1241] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:19:46 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:19:46.486938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:46.516232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:46.588770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480583f7-0e8d-48e4-b97d-00f76640db51 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:46.627275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:46.627872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:46.628128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:46.628401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:46.638499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:46.643871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:46.655315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:46.657179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:46.659532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:46.659532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:46.659532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:46.659532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:46.659532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 615/1242] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:46.664415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:46.759281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:46.764921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e272f83-8f8d-45ab-9f7a-98033e9190d4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:46.799481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:46.800169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:46.800448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:46.800751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:46.801438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:46.801744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:46.803113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:46.814710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 628/1243] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:46.820596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:46.899450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:46.920036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203fbd-c98a-47fd-b49b-145f9abd6696 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:46.953033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:46.953669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:46.953950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:46.954919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:46.961669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:46.963144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:19:46.975634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:46.977039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:46.977984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 616/1244] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:19:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:46.982990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:47.085365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0893d-9cb0-462f-962e-7c3eeb25aa5c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:47.119052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c043d6-b39a-41ff-aa01-7acea4b5eb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:47.119668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c043d6-b39a-41ff-aa01-7acea4b5eb4c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:47.119950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c043d6-b39a-41ff-aa01-7acea4b5eb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:47.120226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c043d6-b39a-41ff-aa01-7acea4b5eb4c None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:47.121006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c043d6-b39a-41ff-aa01-7acea4b5eb4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:47.121318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c043d6-b39a-41ff-aa01-7acea4b5eb4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:47.122747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c043d6-b39a-41ff-aa01-7acea4b5eb4c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:47.135335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 629/1245] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:47.141514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:47.243821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5b7a1-7459-434b-8e97-dee4b2a50fcd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:47.277328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c5017-c5dc-4447-a6df-ce3f0f6e9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:47.277943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5017-c5dc-4447-a6df-ce3f0f6e9a38 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:47.278227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5017-c5dc-4447-a6df-ce3f0f6e9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:47.278496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5017-c5dc-4447-a6df-ce3f0f6e9a38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:47.279275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c5017-c5dc-4447-a6df-ce3f0f6e9a38 None admin] RBAC: Adding query filter params (role.id=224e0c1f1992436bbcf1ba18913cb81b, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=16e8966ae76242898bc996cd039dbe31) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:47.279582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c5017-c5dc-4447-a6df-ce3f0f6e9a38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:47.281022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c5017-c5dc-4447-a6df-ce3f0f6e9a38 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:47.288671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 617/1246] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:19:47 2022] GET /identity/v3/role_assignments?user.id=16e8966ae76242898bc996cd039dbe31&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=224e0c1f1992436bbcf1ba18913cb81b => generated 610 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:49.353074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14247b12-2df8-4042-a395-845db5f0f02d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:49.353609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14247b12-2df8-4042-a395-845db5f0f02d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:49.353609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14247b12-2df8-4042-a395-845db5f0f02d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:49.355007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 630/1247] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:19:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:19:49.359984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da297442-1275-4244-856e-0f713faff440 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:49.361133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da297442-1275-4244-856e-0f713faff440 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:49.361133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da297442-1275-4244-856e-0f713faff440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:49.367414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:49.380287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-da297442-1275-4244-856e-0f713faff440 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:49.381514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da297442-1275-4244-856e-0f713faff440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:49.447828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 618/1248] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:19:49 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:19:49.472028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14247b12-2df8-4042-a395-845db5f0f02d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:49.490686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:49.572687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14247b12-2df8-4042-a395-845db5f0f02d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:49.608064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:49.608672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:49.608954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:49.609230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:49.619153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:19:49.623999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:49.625172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:49.626924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 631/1249] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:19:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 15:19:49.632008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da297442-1275-4244-856e-0f713faff440 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:49.731455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da297442-1275-4244-856e-0f713faff440 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:49.760640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:49.765850 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:49.767155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:49.767515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:49.767799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:49.768552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:49.768871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:49.770389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:49.778347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 619/1250] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:19:49 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:49.784664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:49.862539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:49.885007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062158-242a-48af-9edc-b935a0c76813 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:49.917675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:49.918315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:49.918595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:49.918866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:49.929482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:49.930963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:49.946361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:49.947438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:49.948495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:49.948495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:49.948495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:49.948495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:49.948495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 632/1251] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:49.953333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:50.054076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85630fee-3cad-411f-b4d4-13dd704cd020 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:50.087268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:50.087889 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:50.088188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:50.088484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:50.089185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:50.089496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:50.090880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:50.102989 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 620/1252] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:50.108991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:50.209023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919ed68-8eb7-47f3-9218-bca7365745d2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:50.235918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:50.243458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:50.244098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:50.244378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:50.244680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:50.253085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:50.254514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:19:50.265523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:50.268061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:50.269264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 633/1253] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:50.273749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:50.331637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:50.377595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc41a8-914d-49fc-8362-fb9f66af2437 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:50.412188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:50.412839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:50.413107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:50.413381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:50.414169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:50.414477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:50.415958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:50.434563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 621/1254] 10.222.0.128 () {56 vars in 1148 bytes} [Tue Jul 26 15:19:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:50.440488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:50.539671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843aa61-8062-4a01-951d-04b77ed3b76e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:50.572751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:50.573359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:50.573639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:50.573913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:50.630120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:50.642084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:50.643534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:50.744927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 634/1255] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:19:50 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:19:50.750757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:50.851127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158a9bc-f364-437d-a835-060fbdab4d3e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:50.864179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:50.884715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:50.888193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:50.888193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:50.888193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:50.896481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:50.897923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:50.913496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:50.914500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:50.915540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:50.915540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:50.915540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:50.915540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:50.915540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 622/1256] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:50.920620 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:51.007743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:51.020092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d210-d399-467c-a9cc-54f09315908c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:51.054383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:51.055008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:51.055295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:51.055575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:51.056310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:51.056694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:51.058028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:51.069689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 635/1257] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:19:51.075410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:51.175716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ee241-53f6-4e67-b472-3ef6727cca5c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:51.208942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:51.209577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:51.209859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:51.210131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:51.216596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:51.218025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:19:51.229394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:51.230462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:51.231582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 623/1258] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:19:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:51.236602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:51.338510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a05b8c-e404-418a-8766-7689d33d9480 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:51.378352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41190ac7-630d-4537-b6ee-53b55f1fb706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:51.378972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41190ac7-630d-4537-b6ee-53b55f1fb706 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:51.379257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41190ac7-630d-4537-b6ee-53b55f1fb706 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:51.379538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41190ac7-630d-4537-b6ee-53b55f1fb706 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:51.380299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41190ac7-630d-4537-b6ee-53b55f1fb706 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:51.380777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41190ac7-630d-4537-b6ee-53b55f1fb706 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:51.382225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41190ac7-630d-4537-b6ee-53b55f1fb706 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:51.395117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 636/1259] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:51.401233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:51.427028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:51.508319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4344a1-08ac-49bd-8726-0bd6a963804c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:51.550910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b9531-6f9d-471e-81e0-f5acebea724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:51.551536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b9531-6f9d-471e-81e0-f5acebea724d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:51.551821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b9531-6f9d-471e-81e0-f5acebea724d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:51.552098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b9531-6f9d-471e-81e0-f5acebea724d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:51.552934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b9531-6f9d-471e-81e0-f5acebea724d None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=16e8966ae76242898bc996cd039dbe31) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:51.553247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b9531-6f9d-471e-81e0-f5acebea724d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:51.554697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b9531-6f9d-471e-81e0-f5acebea724d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:51.560376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 624/1260] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:19:51 2022] GET /identity/v3/role_assignments?user.id=16e8966ae76242898bc996cd039dbe31&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:53.794957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cea1b-dfe1-49b4-92dd-82f062b2bafb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:53.795455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cea1b-dfe1-49b4-92dd-82f062b2bafb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:53.795455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cea1b-dfe1-49b4-92dd-82f062b2bafb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:53.796902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 637/1261] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:19:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:19:53.802578 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f1db-9475-47cf-a426-1908d336c35e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:53.802996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f1db-9475-47cf-a426-1908d336c35e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:53.803132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f1db-9475-47cf-a426-1908d336c35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:53.822400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c015f1db-9475-47cf-a426-1908d336c35e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:53.823583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c015f1db-9475-47cf-a426-1908d336c35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:53.890635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 625/1262] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:19:53 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:19:53.912779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cea1b-dfe1-49b4-92dd-82f062b2bafb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:53.924323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:54.016813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cea1b-dfe1-49b4-92dd-82f062b2bafb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:54.051461 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:54.052038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:54.052327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:54.052641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:54.062746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:19:54.068297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:54.069508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:54.070800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 638/1263] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:19:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:54.075496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015f1db-9475-47cf-a426-1908d336c35e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:54.093585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:54.175444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015f1db-9475-47cf-a426-1908d336c35e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:54.210351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:54.210978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:54.211268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:54.211548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:54.212271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:54.212618 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:54.213984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:54.222498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 626/1264] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:19:54 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:54.228757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:54.298045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:54.332120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c3dea-594f-4059-957d-1eeb03af6691 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:54.367246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:54.367885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:54.368168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:54.368445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:54.380132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:54.381618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:54.396332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:54.397480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:54.398665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:54.398665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:54.398665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:54.398665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:54.398665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 639/1265] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:54.403323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:54.497698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:54.516546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9c55d-ccdd-4d0f-9fa3-df6f2bd3da9d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:54.549639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:54.550259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:54.550598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:54.550813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:54.551596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:54.551811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:54.553190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:54.565742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 627/1266] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:19:54.571871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:54.676707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10a7c-349b-4c76-b959-2ee6dc8b0b1d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:54.695813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:54.711472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:54.712231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:54.712556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:54.712842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:54.721164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:54.722623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:19:54.734501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:54.735641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:54.736631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 640/1267] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:54.741623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:54.852055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793338a-899f-40d1-a6ee-98cafb11f4dc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:54.886942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:54.887572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:54.887859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:54.888139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:54.888985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:54.889297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:54.890724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:54.900155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:54.908488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 628/1268] 10.222.0.128 () {56 vars in 1148 bytes} [Tue Jul 26 15:19:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:19:54.914220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:55.017746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88224a0e-60e6-4ff2-8d9f-193cf88dd182 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:55.050303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:55.050940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:55.051222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:55.051499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:55.062117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:55.064336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:55.079070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:55.080174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:55.081442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:55.081442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:55.081442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:55.081442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:55.081442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 641/1269] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:55.086052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:55.188331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048328f8-0750-408e-8663-3a0e9fcfc188 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:55.223048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:55.223669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:55.223954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:55.224235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:55.224957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:55.225268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:55.226898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:55.238835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 629/1270] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:55.244653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:55.276739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:55.346092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410c3c8-4cd6-4339-809d-15897ab715ff None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:55.381359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:55.382010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:55.382291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:55.382571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:55.389284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:55.390728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:19:55.402543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:55.403594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:55.404802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 642/1271] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:19:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:55.410547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:55.470126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:55.511299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d2eca-66ba-40cb-91fb-093da53bfa7e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:55.550523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327e3e5-5959-4e7d-a151-85df253a2adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:55.551065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327e3e5-5959-4e7d-a151-85df253a2adf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:55.551348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327e3e5-5959-4e7d-a151-85df253a2adf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:55.551624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327e3e5-5959-4e7d-a151-85df253a2adf None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:55.552376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327e3e5-5959-4e7d-a151-85df253a2adf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:55.552718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327e3e5-5959-4e7d-a151-85df253a2adf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:55.554310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327e3e5-5959-4e7d-a151-85df253a2adf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:55.566862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 630/1272] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:55.573316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:55.668032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:55.676807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3776-9636-4fa1-b53a-04a13e603187 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:55.710471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901406d-91ad-45b3-a933-4501c28738fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:55.711140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901406d-91ad-45b3-a933-4501c28738fb None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:55.711426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901406d-91ad-45b3-a933-4501c28738fb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:55.711704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901406d-91ad-45b3-a933-4501c28738fb None admin] PATH_INFO: `/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/16e8966ae76242898bc996cd039dbe31/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:55.713722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901406d-91ad-45b3-a933-4501c28738fb None admin] RBAC: Authorizing `identity:create_grant(project_id=95e80fcc351a4e52bbcd7ccdb61baa07, user_id=16e8966ae76242898bc996cd039dbe31, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:55.715167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901406d-91ad-45b3-a933-4501c28738fb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:55.818936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 643/1273] 10.222.0.128 () {58 vars in 1444 bytes} [Tue Jul 26 15:19:55 2022] PUT /identity/v3/projects/95e80fcc351a4e52bbcd7ccdb61baa07/users/16e8966ae76242898bc996cd039dbe31/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:19:57.792348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae895b-d443-4873-bb55-17638ed2e8d9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:57.792934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae895b-d443-4873-bb55-17638ed2e8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:57.792934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae895b-d443-4873-bb55-17638ed2e8d9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:57.794181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 631/1274] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:19:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:19:57.799297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d009ca-de43-41b8-89ad-88328428a35b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:57.799721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d009ca-de43-41b8-89ad-88328428a35b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:57.799863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d009ca-de43-41b8-89ad-88328428a35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:57.818693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a3d009ca-de43-41b8-89ad-88328428a35b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:57.819897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a3d009ca-de43-41b8-89ad-88328428a35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:57.887964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 644/1275] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:19:57 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:19:57.911929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae895b-d443-4873-bb55-17638ed2e8d9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:58.015413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae895b-d443-4873-bb55-17638ed2e8d9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:58.033154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:58.049646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:58.050217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:58.050501 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:58.050776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:58.060703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 15:19:58.066367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:58.067510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:58.068603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 632/1276] 10.222.0.128 () {56 vars in 1101 bytes} [Tue Jul 26 15:19:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:58.073705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d009ca-de43-41b8-89ad-88328428a35b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:58.157218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:58.178375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d009ca-de43-41b8-89ad-88328428a35b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:58.213057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:58.213682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:58.213969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:58.214249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:58.214973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:58.215287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:58.216692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:58.224940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 645/1277] 10.222.0.128 () {56 vars in 1104 bytes} [Tue Jul 26 15:19:58 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:58.230902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:58.331495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b88ac-3285-4e39-99e4-c2ba80826a50 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:58.365293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:58.365941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:58.366217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:58.366494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:58.378182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:58.379628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:58.394309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:58.395418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:58.396713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:58.396713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:58.396713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:58.396713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:58.396713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 633/1278] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:58.401232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:58.504613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b7d2-c2df-475a-9ceb-e703f504671e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:58.538603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:58.539230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:58.539513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:58.539789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:58.540516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:58.540826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:58.542434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:58.554524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 646/1279] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:58.560253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:58.591711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:58.660995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356aa4e0-d115-4b22-96c3-e8c8f2c0ac5a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:58.693367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:58.694004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:58.694284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:58.694558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:58.702077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:58.704287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 15:19:58.716520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:58.717787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:19:58.719181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 634/1280] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:58.723634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:58.727756 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:58.823904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771e784-1054-42e2-a3ea-620603913699 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:58.858286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:58.858834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:58.859119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:58.859444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:58.860557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:58.860782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:58.862273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:58.879824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 647/1281] 10.222.0.128 () {56 vars in 1148 bytes} [Tue Jul 26 15:19:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:58.885620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:58.977790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:58.988218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fe3cb-2300-407b-a1be-b2118148df00 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:59.021733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:59.022360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:59.022646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:59.022929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:59.090195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:59.091634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:59.180288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:59.190854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 635/1282] 10.222.0.128 () {58 vars in 1287 bytes} [Tue Jul 26 15:19:58 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 15:19:59.196856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:59.264545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:59.302723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4669de6-5ebc-4cf8-997e-80c313d50808 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:59.337309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:59.337948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:59.338231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:59.338509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:59.348429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:59.349910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 15:19:59.366395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:59.373184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:59.374299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 15:19:59.374299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 15:19:59.374299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 15:19:59.374299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:59.374299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 648/1283] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:59.374673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:59.475351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123a9ba-36c3-46b7-a1a4-911fe52fa0d6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:59.509810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:59.510427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:59.510721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:59.510992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:59.511687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:59.511993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:59.513390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:59.525604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 636/1284] 10.222.0.128 () {56 vars in 1114 bytes} [Tue Jul 26 15:19:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:19:59.532785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:59.637991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7168d6a-cd92-428a-a351-bde6dfaadc02 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:59.653681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:59.674936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:59.675580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:59.675864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:59.676142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:59.682770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:59.684208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 15:19:59.696053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:19:59.699687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 26 15:19:59.700861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 649/1285] 10.222.0.128 () {56 vars in 1116 bytes} [Tue Jul 26 15:19:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:19:59.704804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:59.755650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:59.805846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fb724-daa3-48ca-89a7-48a25dfb1259 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:59.839365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c462783-e7ce-41be-858b-f9871abf74fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:59.839975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c462783-e7ce-41be-858b-f9871abf74fd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:59.840262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c462783-e7ce-41be-858b-f9871abf74fd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:59.840563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c462783-e7ce-41be-858b-f9871abf74fd None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:59.841321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c462783-e7ce-41be-858b-f9871abf74fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:19:59.841632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c462783-e7ce-41be-858b-f9871abf74fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:59.843058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c462783-e7ce-41be-858b-f9871abf74fd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:59.857144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 637/1286] 10.222.0.128 () {56 vars in 1153 bytes} [Tue Jul 26 15:19:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:19:59.863694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:59.969709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1a496-d848-41f4-9d1d-91901d55ea54 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:00.004273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb2554-ae60-444c-a2a9-77bee430a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:00.004915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2554-ae60-444c-a2a9-77bee430a7db None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:00.005201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2554-ae60-444c-a2a9-77bee430a7db None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:00.005479 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2554-ae60-444c-a2a9-77bee430a7db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:00.006315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2554-ae60-444c-a2a9-77bee430a7db None admin] RBAC: Adding query filter params (role.id=846ff459eaba4359b0bc2e143d793bfd, scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07, user.id=16e8966ae76242898bc996cd039dbe31) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:20:00.006581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2554-ae60-444c-a2a9-77bee430a7db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:00.008018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2554-ae60-444c-a2a9-77bee430a7db None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:00.013775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 650/1287] 10.222.0.128 () {56 vars in 1379 bytes} [Tue Jul 26 15:19:59 2022] GET /identity/v3/role_assignments?user.id=16e8966ae76242898bc996cd039dbe31&scope.project.id=95e80fcc351a4e52bbcd7ccdb61baa07&role.id=846ff459eaba4359b0bc2e143d793bfd => generated 610 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:20:02.063167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c8ef3-5ecf-49b5-a837-2ccde461bf7c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:02.063702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c8ef3-5ecf-49b5-a837-2ccde461bf7c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:02.063814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c8ef3-5ecf-49b5-a837-2ccde461bf7c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:02.065132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 638/1288] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:20:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:20:02.070354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c4cb3-0c9c-41c4-a5a3-33d225e7ea84 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:02.070770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c4cb3-0c9c-41c4-a5a3-33d225e7ea84 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:02.070919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c4cb3-0c9c-41c4-a5a3-33d225e7ea84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:02.090203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0c7c4cb3-0c9c-41c4-a5a3-33d225e7ea84 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:20:02.091402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0c7c4cb3-0c9c-41c4-a5a3-33d225e7ea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:20:02.121782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:02.159845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 651/1289] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:20:02 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:20:02.181998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c8ef3-5ecf-49b5-a837-2ccde461bf7c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:02.286149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c8ef3-5ecf-49b5-a837-2ccde461bf7c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:02.296432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:02.319398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:02.319988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:02.320273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:02.320582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:02.321216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:02.322578 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:20:02.332154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:20:02.337391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:02.338470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 639/1290] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:20:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:20:02.339603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c4cb3-0c9c-41c4-a5a3-33d225e7ea84 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:02.440029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c4cb3-0c9c-41c4-a5a3-33d225e7ea84 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:02.474740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38546252-0253-4ff9-911b-088d0c5e1f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:02.475366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546252-0253-4ff9-911b-088d0c5e1f32 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:02.475918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546252-0253-4ff9-911b-088d0c5e1f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:02.476199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546252-0253-4ff9-911b-088d0c5e1f32 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:02.476917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38546252-0253-4ff9-911b-088d0c5e1f32 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:20:02.477228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38546252-0253-4ff9-911b-088d0c5e1f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:02.478596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38546252-0253-4ff9-911b-088d0c5e1f32 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:02.483374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 652/1291] 10.222.0.128 () {56 vars in 1121 bytes} [Tue Jul 26 15:20:02 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:20:02.489246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:02.591041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c7d07-5f3b-418d-b105-9ada3e594036 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:02.626961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6acb6-94ca-41d7-bc10-966c4aa12de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:02.627577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6acb6-94ca-41d7-bc10-966c4aa12de2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:02.627899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6acb6-94ca-41d7-bc10-966c4aa12de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:02.628175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6acb6-94ca-41d7-bc10-966c4aa12de2 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:02.628888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6acb6-94ca-41d7-bc10-966c4aa12de2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:20:02.629194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6acb6-94ca-41d7-bc10-966c4aa12de2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:02.630549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6acb6-94ca-41d7-bc10-966c4aa12de2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:02.635036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 640/1292] 10.222.0.128 () {56 vars in 1121 bytes} [Tue Jul 26 15:20:02 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:20:04.647674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c043c11-2fb3-4308-abec-b19bc1be4c75 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:04.648247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c043c11-2fb3-4308-abec-b19bc1be4c75 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:04.648247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c043c11-2fb3-4308-abec-b19bc1be4c75 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:04.649638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 653/1293] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:20:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:20:04.654879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ece86-c870-4983-b3c5-5f503a3cbbb7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:04.656409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ece86-c870-4983-b3c5-5f503a3cbbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:04.656409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ece86-c870-4983-b3c5-5f503a3cbbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:04.683657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ee4ece86-c870-4983-b3c5-5f503a3cbbb7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:20:04.684900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee4ece86-c870-4983-b3c5-5f503a3cbbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:20:04.714571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:04.751869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 641/1294] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:20:04 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 15:20:04.774479 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c043c11-2fb3-4308-abec-b19bc1be4c75 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:04.806482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:04.876758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c043c11-2fb3-4308-abec-b19bc1be4c75 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:04.911575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf9fb4-a8af-4384-aea8-584cd1435281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:04.912197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf9fb4-a8af-4384-aea8-584cd1435281 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:04.912504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf9fb4-a8af-4384-aea8-584cd1435281 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:04.912826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf9fb4-a8af-4384-aea8-584cd1435281 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:04.913516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf9fb4-a8af-4384-aea8-584cd1435281 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:04.914893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf9fb4-a8af-4384-aea8-584cd1435281 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:04.981121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 654/1295] 10.222.0.128 () {60 vars in 1146 bytes} [Tue Jul 26 15:20:04 2022] POST /identity/v3/services => generated 247 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:20:07.007659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0088d-4f30-485b-aded-f8aedce17db6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:07.008278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0088d-4f30-485b-aded-f8aedce17db6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:07.008278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0088d-4f30-485b-aded-f8aedce17db6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:07.009628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 642/1296] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:20:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:20:07.015605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147c6ab-93fd-4e71-b703-0996acffd874 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:07.016147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147c6ab-93fd-4e71-b703-0996acffd874 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:07.016244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147c6ab-93fd-4e71-b703-0996acffd874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:07.034075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4147c6ab-93fd-4e71-b703-0996acffd874 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:20:07.035253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4147c6ab-93fd-4e71-b703-0996acffd874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:20:07.111223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 655/1297] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:20:07 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:20:07.133530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0088d-4f30-485b-aded-f8aedce17db6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:07.237680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0088d-4f30-485b-aded-f8aedce17db6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:07.272527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:07.273126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:07.273411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:07.273687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:07.274315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:07.275683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:20:07.285237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:20:07.287168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:07.288870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 643/1298] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:20:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:20:07.292526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147c6ab-93fd-4e71-b703-0996acffd874 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:07.310410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:07.390796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147c6ab-93fd-4e71-b703-0996acffd874 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:07.423839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:07.424458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:07.424770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:07.425092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:07.425774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:20:07.426083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:07.427476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:07.432437 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 656/1299] 10.222.0.128 () {56 vars in 1121 bytes} [Tue Jul 26 15:20:07 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:20:07.438729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:07.472211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:07.541638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f96de4-5c5f-4c43-bc5c-1948daca34ad None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:07.574427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5cd00-f814-40bb-be60-8ac3f316068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:07.575047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5cd00-f814-40bb-be60-8ac3f316068f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:07.575334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5cd00-f814-40bb-be60-8ac3f316068f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:07.575612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5cd00-f814-40bb-be60-8ac3f316068f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:07.576369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5cd00-f814-40bb-be60-8ac3f316068f None admin] RBAC: Adding query filter params (interface=public, service_id=61ab80acd9a549ce934537ce50fc271a, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:20:07.576714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5cd00-f814-40bb-be60-8ac3f316068f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:07.578698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5cd00-f814-40bb-be60-8ac3f316068f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:07.583684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 644/1300] 10.222.0.128 () {56 vars in 1256 bytes} [Tue Jul 26 15:20:07 2022] GET /identity/v3/endpoints?service_id=61ab80acd9a549ce934537ce50fc271a&interface=public®ion_id=RegionOne => generated 195 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:20:07.589622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:07.672434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:07.690827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464ecbe-2131-40d0-8552-355f04c91849 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:07.724929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a3492-fb7a-4237-87e9-b1d58922ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:07.725811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a3492-fb7a-4237-87e9-b1d58922ebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:07.726092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a3492-fb7a-4237-87e9-b1d58922ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:07.726366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a3492-fb7a-4237-87e9-b1d58922ebe4 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:07.727014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a3492-fb7a-4237-87e9-b1d58922ebe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:07.728377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a3492-fb7a-4237-87e9-b1d58922ebe4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:07.735056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 657/1301] 10.222.0.128 () {56 vars in 1092 bytes} [Tue Jul 26 15:20:07 2022] GET /identity/v3/services => generated 2201 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:20:09.710838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a258-e4e6-4cb1-a3d1-03c710633151 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:09.711369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a258-e4e6-4cb1-a3d1-03c710633151 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:09.711369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a258-e4e6-4cb1-a3d1-03c710633151 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:09.712704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 645/1302] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:20:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:20:09.717675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377cb9f-4f4f-4583-8d06-34c44408f08b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:09.719513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377cb9f-4f4f-4583-8d06-34c44408f08b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:09.719513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377cb9f-4f4f-4583-8d06-34c44408f08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:09.736794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0377cb9f-4f4f-4583-8d06-34c44408f08b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:20:09.737964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0377cb9f-4f4f-4583-8d06-34c44408f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:20:09.804627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 658/1303] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:20:09 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 15:20:09.827130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96a258-e4e6-4cb1-a3d1-03c710633151 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:09.924847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:09.931544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96a258-e4e6-4cb1-a3d1-03c710633151 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:09.967658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:09.968382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:09.968850 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:09.969128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:09.969755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:09.971226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 15:20:09.980664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 15:20:09.981864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jul 26 15:20:09.982739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 646/1304] 10.222.0.128 () {56 vars in 1122 bytes} [Tue Jul 26 15:20:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 15:20:09.988390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377cb9f-4f4f-4583-8d06-34c44408f08b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:10.066928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:10.090367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377cb9f-4f4f-4583-8d06-34c44408f08b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:10.126468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a7d25-38df-4a02-a5f8-47fdb9a87029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:10.127089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a7d25-38df-4a02-a5f8-47fdb9a87029 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:10.127371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a7d25-38df-4a02-a5f8-47fdb9a87029 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:10.127648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a7d25-38df-4a02-a5f8-47fdb9a87029 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:10.128329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a7d25-38df-4a02-a5f8-47fdb9a87029 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:20:10.128665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a7d25-38df-4a02-a5f8-47fdb9a87029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:10.130089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a7d25-38df-4a02-a5f8-47fdb9a87029 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:10.135150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 659/1305] 10.222.0.128 () {56 vars in 1121 bytes} [Tue Jul 26 15:20:09 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:20:10.142030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:10.242575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69b2d-1d1a-488b-8536-fc3b8b7c10b0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:10.276976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511e98b-4639-460c-8a10-319b21adbeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:10.277591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511e98b-4639-460c-8a10-319b21adbeb4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:10.277871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511e98b-4639-460c-8a10-319b21adbeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:10.278145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511e98b-4639-460c-8a10-319b21adbeb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:10.278849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511e98b-4639-460c-8a10-319b21adbeb4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:10.280203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511e98b-4639-460c-8a10-319b21adbeb4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:10.381878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 647/1306] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:20:10 2022] POST /identity/v3/endpoints => generated 333 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:20:20.132097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=30 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:20.377042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:18.829300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2596f-e177-4601-bac2-a3c9fed26767 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:18.830337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2596f-e177-4601-bac2-a3c9fed26767 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:18.830864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2596f-e177-4601-bac2-a3c9fed26767 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:18.836434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 660/1307] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:18 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:18.837828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b14860-841d-43b9-8dc3-e85dc8986caf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:18.838244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b14860-841d-43b9-8dc3-e85dc8986caf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:18.838446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b14860-841d-43b9-8dc3-e85dc8986caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:19.048271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e0b14860-841d-43b9-8dc3-e85dc8986caf None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:19.051775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e0b14860-841d-43b9-8dc3-e85dc8986caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:19.231976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 648/1308] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:26:19.301483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09552a-800f-4e53-b959-5b5ce3d63336 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:19.302782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09552a-800f-4e53-b959-5b5ce3d63336 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:19.303239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09552a-800f-4e53-b959-5b5ce3d63336 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:19.305076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 661/1309] 10.222.0.128 () {54 vars in 970 bytes} [Tue Jul 26 15:26:19 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:19.318945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488ff6e-b9c5-47cf-bd7e-53569d32753f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:19.320275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488ff6e-b9c5-47cf-bd7e-53569d32753f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:19.320832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488ff6e-b9c5-47cf-bd7e-53569d32753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:19.422705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b488ff6e-b9c5-47cf-bd7e-53569d32753f None None] MFA Rules not processed for user `c6ecc741022f4d6189f4d3f19b4cdbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:19.423865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b488ff6e-b9c5-47cf-bd7e-53569d32753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:19.448959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:19.520456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 649/1310] 10.222.0.128 () {58 vars in 1067 bytes} [Tue Jul 26 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:19.529291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09552a-800f-4e53-b959-5b5ce3d63336 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:19.531965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ba09552a-800f-4e53-b959-5b5ce3d63336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:19.659481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09552a-800f-4e53-b959-5b5ce3d63336 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:19.754307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90956c4d-05ad-444c-b841-b61ec312ac97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:19.755203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956c4d-05ad-444c-b841-b61ec312ac97 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:19.755633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956c4d-05ad-444c-b841-b61ec312ac97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:19.756082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956c4d-05ad-444c-b841-b61ec312ac97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:19.757976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90956c4d-05ad-444c-b841-b61ec312ac97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:19.826170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:19.897449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90956c4d-05ad-444c-b841-b61ec312ac97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:19.898928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90956c4d-05ad-444c-b841-b61ec312ac97 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:19.949077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 662/1311] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:22.400453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bd9a-a710-4fc4-9b62-fc6fc23ac866 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:22.401010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bd9a-a710-4fc4-9b62-fc6fc23ac866 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:22.401010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bd9a-a710-4fc4-9b62-fc6fc23ac866 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:22.402362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 650/1312] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:22.408042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8afb53-edcc-4b28-93d0-6d684ed12f5a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:22.408490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8afb53-edcc-4b28-93d0-6d684ed12f5a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:22.408776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8afb53-edcc-4b28-93d0-6d684ed12f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:22.490096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dd8afb53-edcc-4b28-93d0-6d684ed12f5a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:22.491361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dd8afb53-edcc-4b28-93d0-6d684ed12f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:22.602923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 663/1313] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:22.642632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51bd9a-a710-4fc4-9b62-fc6fc23ac866 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:22.644241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6a51bd9a-a710-4fc4-9b62-fc6fc23ac866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:22.755796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51bd9a-a710-4fc4-9b62-fc6fc23ac866 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:22.829927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:22.832457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7631b-4d9b-489c-90f2-0aeeffe6e809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:22.833105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7631b-4d9b-489c-90f2-0aeeffe6e809 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:22.833388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7631b-4d9b-489c-90f2-0aeeffe6e809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:22.833666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7631b-4d9b-489c-90f2-0aeeffe6e809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:22.835328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-93a7631b-4d9b-489c-90f2-0aeeffe6e809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:22.959212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7631b-4d9b-489c-90f2-0aeeffe6e809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:22.960685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7631b-4d9b-489c-90f2-0aeeffe6e809 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:22.978893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 651/1314] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:25.465245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ee116-3112-4935-9700-f30462ca080f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:25.465675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ee116-3112-4935-9700-f30462ca080f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:25.465800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ee116-3112-4935-9700-f30462ca080f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:25.467676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 664/1315] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:25.472892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3f4a-e561-4271-9907-da2271bf5475 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:25.473310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3f4a-e561-4271-9907-da2271bf5475 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:25.473452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3f4a-e561-4271-9907-da2271bf5475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:25.539010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3e6f3f4a-e561-4271-9907-da2271bf5475 None None] MFA Rules not processed for user `af6d7c3cdb6c4cc9a4f4f8447cb9227c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:25.540225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e6f3f4a-e561-4271-9907-da2271bf5475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:25.591331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:25.636099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 652/1316] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:25.675886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ee116-3112-4935-9700-f30462ca080f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:25.677459 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-319ee116-3112-4935-9700-f30462ca080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:25.688435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:25.783703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ee116-3112-4935-9700-f30462ca080f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:25.856523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea59025-de90-4d89-8b1b-b3b8a8f1f866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:25.857145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea59025-de90-4d89-8b1b-b3b8a8f1f866 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:25.858042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea59025-de90-4d89-8b1b-b3b8a8f1f866 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:25.858319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea59025-de90-4d89-8b1b-b3b8a8f1f866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:25.859965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4ea59025-de90-4d89-8b1b-b3b8a8f1f866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:25.921960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:25.968975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea59025-de90-4d89-8b1b-b3b8a8f1f866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:25.970420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea59025-de90-4d89-8b1b-b3b8a8f1f866 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:25.989253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 665/1317] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:28.879195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071625f9-2571-402f-8fda-9885c4ea31fb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:28.879642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071625f9-2571-402f-8fda-9885c4ea31fb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:28.879756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071625f9-2571-402f-8fda-9885c4ea31fb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:28.881143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 653/1318] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:28.886199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bdc1f-6361-4316-bbaf-9b58c0920d44 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:28.886630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bdc1f-6361-4316-bbaf-9b58c0920d44 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:28.886763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bdc1f-6361-4316-bbaf-9b58c0920d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:28.953646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-030bdc1f-6361-4316-bbaf-9b58c0920d44 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:28.960303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-030bdc1f-6361-4316-bbaf-9b58c0920d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:29.052777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 666/1319] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:29.091925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071625f9-2571-402f-8fda-9885c4ea31fb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:29.093667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-071625f9-2571-402f-8fda-9885c4ea31fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:29.207370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071625f9-2571-402f-8fda-9885c4ea31fb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:29.284090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8419c50-b9f2-4ebe-9fb0-1073fd7e1948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:29.284750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8419c50-b9f2-4ebe-9fb0-1073fd7e1948 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:29.285035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8419c50-b9f2-4ebe-9fb0-1073fd7e1948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:29.285310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8419c50-b9f2-4ebe-9fb0-1073fd7e1948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:29.286953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e8419c50-b9f2-4ebe-9fb0-1073fd7e1948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:29.295312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:29.402290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8419c50-b9f2-4ebe-9fb0-1073fd7e1948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:29.403797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8419c50-b9f2-4ebe-9fb0-1073fd7e1948 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:29.422939 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 654/1320] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:32.425882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db3c5-c0ac-4411-ab8c-25f61c57fd8b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:32.426457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db3c5-c0ac-4411-ab8c-25f61c57fd8b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:32.426457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db3c5-c0ac-4411-ab8c-25f61c57fd8b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:32.427800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 667/1321] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:32.432981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab72bf1-32a5-4828-9399-271620804859 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:32.433397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab72bf1-32a5-4828-9399-271620804859 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:32.433551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab72bf1-32a5-4828-9399-271620804859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:32.505740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9ab72bf1-32a5-4828-9399-271620804859 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:32.506946 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9ab72bf1-32a5-4828-9399-271620804859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:32.563905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:32.604667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 655/1322] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:32.642655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5db3c5-c0ac-4411-ab8c-25f61c57fd8b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:32.644199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4b5db3c5-c0ac-4411-ab8c-25f61c57fd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:32.664774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:32.760278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5db3c5-c0ac-4411-ab8c-25f61c57fd8b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:32.845800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429ab2c-e331-4a99-bfe8-94130de0b025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:32.846431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ab2c-e331-4a99-bfe8-94130de0b025 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:32.846708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ab2c-e331-4a99-bfe8-94130de0b025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:32.846982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ab2c-e331-4a99-bfe8-94130de0b025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:32.848704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6429ab2c-e331-4a99-bfe8-94130de0b025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:32.929798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:32.969549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ab2c-e331-4a99-bfe8-94130de0b025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:32.970996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ab2c-e331-4a99-bfe8-94130de0b025 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:32.991226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 668/1323] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:35.096819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f8302-364d-4fff-a481-6b402567721a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:35.097402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f8302-364d-4fff-a481-6b402567721a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:35.097402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f8302-364d-4fff-a481-6b402567721a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:35.098721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 656/1324] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:35.104148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75d3aa-b273-4634-97b3-d60ad341192a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:35.104568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75d3aa-b273-4634-97b3-d60ad341192a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:35.104733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75d3aa-b273-4634-97b3-d60ad341192a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:35.173075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cf75d3aa-b273-4634-97b3-d60ad341192a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:35.174202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cf75d3aa-b273-4634-97b3-d60ad341192a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:35.277201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 669/1325] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:35.329519 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f983b-e693-452c-9e28-ebf9886a24d3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:35.329802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f983b-e693-452c-9e28-ebf9886a24d3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:35.330080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f983b-e693-452c-9e28-ebf9886a24d3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:35.331426 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 657/1326] 10.222.0.128 () {54 vars in 970 bytes} [Tue Jul 26 15:26:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:35.337519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd6e11-6ee1-4a86-8325-96e23b9ab5fb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:35.337822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd6e11-6ee1-4a86-8325-96e23b9ab5fb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:35.338105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd6e11-6ee1-4a86-8325-96e23b9ab5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:35.370398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:35.415156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-31cd6e11-6ee1-4a86-8325-96e23b9ab5fb None None] MFA Rules not processed for user `c6ecc741022f4d6189f4d3f19b4cdbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:35.416363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-31cd6e11-6ee1-4a86-8325-96e23b9ab5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:35.495875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 670/1327] 10.222.0.128 () {58 vars in 1067 bytes} [Tue Jul 26 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:35.505884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f983b-e693-452c-9e28-ebf9886a24d3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:35.507434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d38f983b-e693-452c-9e28-ebf9886a24d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:35.617116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f983b-e693-452c-9e28-ebf9886a24d3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:35.696458 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcf9a0-d6bf-4c42-ae6a-7e82743ea2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:35.697102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcf9a0-d6bf-4c42-ae6a-7e82743ea2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:35.697385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcf9a0-d6bf-4c42-ae6a-7e82743ea2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:35.697656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcf9a0-d6bf-4c42-ae6a-7e82743ea2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:35.699293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-01dcf9a0-d6bf-4c42-ae6a-7e82743ea2a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:35.716214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:35.818025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcf9a0-d6bf-4c42-ae6a-7e82743ea2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:35.819476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcf9a0-d6bf-4c42-ae6a-7e82743ea2a0 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:35.838943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 658/1328] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:38.187010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9d070-c7e8-4372-b21a-f2ffe463a76c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:38.187587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9d070-c7e8-4372-b21a-f2ffe463a76c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:38.187587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9d070-c7e8-4372-b21a-f2ffe463a76c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:38.189079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 671/1329] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:38.194398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839825b0-9887-4e7c-802e-d42e12e596cd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:38.194836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839825b0-9887-4e7c-802e-d42e12e596cd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:38.194961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839825b0-9887-4e7c-802e-d42e12e596cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:38.263762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-839825b0-9887-4e7c-802e-d42e12e596cd None None] MFA Rules not processed for user `af6d7c3cdb6c4cc9a4f4f8447cb9227c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:38.265005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-839825b0-9887-4e7c-802e-d42e12e596cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:38.333302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:38.367445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 659/1330] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:38.406028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9d070-c7e8-4372-b21a-f2ffe463a76c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:38.407633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-02d9d070-c7e8-4372-b21a-f2ffe463a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:38.535527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9d070-c7e8-4372-b21a-f2ffe463a76c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:38.557485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:38.616039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c54a6-71d0-43a8-aa78-da02b13bbde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:38.616706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c54a6-71d0-43a8-aa78-da02b13bbde1 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:38.616991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c54a6-71d0-43a8-aa78-da02b13bbde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:38.617269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c54a6-71d0-43a8-aa78-da02b13bbde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:38.618907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e61c54a6-71d0-43a8-aa78-da02b13bbde1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:38.730317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c54a6-71d0-43a8-aa78-da02b13bbde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:38.731721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c54a6-71d0-43a8-aa78-da02b13bbde1 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:38.752280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 672/1331] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:41.439078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89508-63e5-4ca2-9de0-38f7c6193353 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:41.439614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89508-63e5-4ca2-9de0-38f7c6193353 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:41.439739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89508-63e5-4ca2-9de0-38f7c6193353 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:41.441031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 660/1332] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:41.446290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fa594-35f7-4e2f-95d3-da49d5d38d84 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:41.446671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fa594-35f7-4e2f-95d3-da49d5d38d84 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:41.446847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fa594-35f7-4e2f-95d3-da49d5d38d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:41.511727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:41.514640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-003fa594-35f7-4e2f-95d3-da49d5d38d84 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:41.515849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-003fa594-35f7-4e2f-95d3-da49d5d38d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:41.614024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 673/1333] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:41.653391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae89508-63e5-4ca2-9de0-38f7c6193353 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:41.655500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cae89508-63e5-4ca2-9de0-38f7c6193353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:41.757747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae89508-63e5-4ca2-9de0-38f7c6193353 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:41.829584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2c2dd-1f0e-4f08-a2e3-eb6c8df36bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:41.830199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2c2dd-1f0e-4f08-a2e3-eb6c8df36bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:41.830480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2c2dd-1f0e-4f08-a2e3-eb6c8df36bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:41.830753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2c2dd-1f0e-4f08-a2e3-eb6c8df36bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:41.832356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-12f2c2dd-1f0e-4f08-a2e3-eb6c8df36bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:41.859507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:41.940885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2c2dd-1f0e-4f08-a2e3-eb6c8df36bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:41.942323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2c2dd-1f0e-4f08-a2e3-eb6c8df36bbb service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:41.960591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 661/1334] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:44.041258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97215-de5f-49d3-b00a-1fd241249143 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:44.041899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97215-de5f-49d3-b00a-1fd241249143 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:44.041899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97215-de5f-49d3-b00a-1fd241249143 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:44.043188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 674/1335] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:44.049503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15fba-af24-4512-99d9-64e0f382c5f9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:44.049942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15fba-af24-4512-99d9-64e0f382c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:44.050040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15fba-af24-4512-99d9-64e0f382c5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:44.117022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c6e15fba-af24-4512-99d9-64e0f382c5f9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:44.118231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c6e15fba-af24-4512-99d9-64e0f382c5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:44.193607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:44.215759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 662/1336] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:44.259652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b97215-de5f-49d3-b00a-1fd241249143 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:44.261224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-41b97215-de5f-49d3-b00a-1fd241249143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:44.365907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b97215-de5f-49d3-b00a-1fd241249143 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:44.393537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:44.439283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfd3b4-a7af-4ad6-895f-d05099793d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:44.439899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd3b4-a7af-4ad6-895f-d05099793d86 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:44.440181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd3b4-a7af-4ad6-895f-d05099793d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:44.440458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd3b4-a7af-4ad6-895f-d05099793d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:44.442112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7ddfd3b4-a7af-4ad6-895f-d05099793d86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:44.553778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfd3b4-a7af-4ad6-895f-d05099793d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:44.555242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfd3b4-a7af-4ad6-895f-d05099793d86 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:44.573639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 675/1337] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:47.409317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53265f0e-429c-442e-8c86-3e1ab155b605 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:47.409887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53265f0e-429c-442e-8c86-3e1ab155b605 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:47.409887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53265f0e-429c-442e-8c86-3e1ab155b605 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:47.411194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 663/1338] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:47.416873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b7d0-3b01-469f-a29b-65d6620afcdd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:47.417272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b7d0-3b01-469f-a29b-65d6620afcdd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:47.417468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b7d0-3b01-469f-a29b-65d6620afcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:47.485466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b287b7d0-3b01-469f-a29b-65d6620afcdd None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:47.486699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b287b7d0-3b01-469f-a29b-65d6620afcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:47.494961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:47.583992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 676/1339] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:26:47.621248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53265f0e-429c-442e-8c86-3e1ab155b605 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:47.622860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-53265f0e-429c-442e-8c86-3e1ab155b605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:47.728006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53265f0e-429c-442e-8c86-3e1ab155b605 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:47.800861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cbd65-7fd1-4da9-bee6-a16aca6fd3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:47.801481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cbd65-7fd1-4da9-bee6-a16aca6fd3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:47.801768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cbd65-7fd1-4da9-bee6-a16aca6fd3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:47.802048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cbd65-7fd1-4da9-bee6-a16aca6fd3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:47.803675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-650cbd65-7fd1-4da9-bee6-a16aca6fd3b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:47.838775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:47.913435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cbd65-7fd1-4da9-bee6-a16aca6fd3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:47.914897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cbd65-7fd1-4da9-bee6-a16aca6fd3b3 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:47.933179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 664/1340] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:52.249550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec588e-5038-4446-942f-815273930195 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:52.250133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec588e-5038-4446-942f-815273930195 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:52.250133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec588e-5038-4446-942f-815273930195 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:52.251907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 677/1341] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:52.257209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c1a56-89a3-4a65-8788-a2dacc8e7603 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:52.257635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c1a56-89a3-4a65-8788-a2dacc8e7603 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:52.257737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c1a56-89a3-4a65-8788-a2dacc8e7603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:52.336210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a20c1a56-89a3-4a65-8788-a2dacc8e7603 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:52.338075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a20c1a56-89a3-4a65-8788-a2dacc8e7603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:52.429339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:52.441460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 665/1342] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:52.481702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec588e-5038-4446-942f-815273930195 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:52.483490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e9ec588e-5038-4446-942f-815273930195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:52.591041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec588e-5038-4446-942f-815273930195 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:52.630384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:52.668669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ae6b4-55bc-4b6a-bffa-7878dcdad3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:52.669292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ae6b4-55bc-4b6a-bffa-7878dcdad3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:52.669574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ae6b4-55bc-4b6a-bffa-7878dcdad3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:52.670447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ae6b4-55bc-4b6a-bffa-7878dcdad3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:52.672258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c81ae6b4-55bc-4b6a-bffa-7878dcdad3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:52.784966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ae6b4-55bc-4b6a-bffa-7878dcdad3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:52.786322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ae6b4-55bc-4b6a-bffa-7878dcdad3f4 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:52.804884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 678/1343] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:26:55.847739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebde2e-b18a-4605-8317-1e0a27eefc95 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:55.848301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebde2e-b18a-4605-8317-1e0a27eefc95 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:55.848301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebde2e-b18a-4605-8317-1e0a27eefc95 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:55.849685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 666/1344] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:26:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:26:55.855217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c8f98-64f8-48a2-8512-05b634c59ef5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:55.855668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c8f98-64f8-48a2-8512-05b634c59ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:55.855904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c8f98-64f8-48a2-8512-05b634c59ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:55.922388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3f2c8f98-64f8-48a2-8512-05b634c59ef5 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:26:55.923610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3f2c8f98-64f8-48a2-8512-05b634c59ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:55.941862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:56.022593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 679/1345] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:26:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:26:56.061673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebde2e-b18a-4605-8317-1e0a27eefc95 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:26:56.063302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ceebde2e-b18a-4605-8317-1e0a27eefc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:56.170882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebde2e-b18a-4605-8317-1e0a27eefc95 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:26:56.246576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1b4c5-c5d2-4102-8c1f-1441e9c48eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:26:56.247477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b4c5-c5d2-4102-8c1f-1441e9c48eed service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:26:56.247791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b4c5-c5d2-4102-8c1f-1441e9c48eed service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:26:56.248139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b4c5-c5d2-4102-8c1f-1441e9c48eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:26:56.249935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-abd1b4c5-c5d2-4102-8c1f-1441e9c48eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:26:56.294061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:26:56.365878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1b4c5-c5d2-4102-8c1f-1441e9c48eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:26:56.367367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1b4c5-c5d2-4102-8c1f-1441e9c48eed service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:26:56.386392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 667/1346] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:00.762690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec5f6c-2346-4871-a0b1-35f1d949f5ad None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:00.763867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec5f6c-2346-4871-a0b1-35f1d949f5ad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:00.763867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec5f6c-2346-4871-a0b1-35f1d949f5ad None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:00.765382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 680/1347] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:00.770957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249283b-de44-497d-857f-cadc6bf353f4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:00.771428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249283b-de44-497d-857f-cadc6bf353f4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:00.771509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249283b-de44-497d-857f-cadc6bf353f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:00.845138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d249283b-de44-497d-857f-cadc6bf353f4 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:00.846369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d249283b-de44-497d-857f-cadc6bf353f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:00.987425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:00.990226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 668/1348] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:01.030777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec5f6c-2346-4871-a0b1-35f1d949f5ad None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:01.032889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-beec5f6c-2346-4871-a0b1-35f1d949f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:01.166117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec5f6c-2346-4871-a0b1-35f1d949f5ad None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:01.218696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:01.247413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c407383-bf4e-49d8-9972-952d9faf2ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:01.249015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407383-bf4e-49d8-9972-952d9faf2ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:01.249212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407383-bf4e-49d8-9972-952d9faf2ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:01.250214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407383-bf4e-49d8-9972-952d9faf2ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:01.251473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7c407383-bf4e-49d8-9972-952d9faf2ed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:01.374102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407383-bf4e-49d8-9972-952d9faf2ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:01.376101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407383-bf4e-49d8-9972-952d9faf2ed1 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:01.397764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 681/1349] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:27:06.344531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19624-fadf-43ca-b827-c669c8d2afb8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:06.345335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19624-fadf-43ca-b827-c669c8d2afb8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:06.345830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19624-fadf-43ca-b827-c669c8d2afb8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:06.347351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 669/1350] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:27:06 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:06.355764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9273521-9257-48db-ad1f-52cb0de95ab6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:06.355764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9273521-9257-48db-ad1f-52cb0de95ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:06.355764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9273521-9257-48db-ad1f-52cb0de95ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:06.507757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c9273521-9257-48db-ad1f-52cb0de95ab6 None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:06.509023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c9273521-9257-48db-ad1f-52cb0de95ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:06.537281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:06.643851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 682/1351] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:10.765832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad417-f483-4d9e-8fd5-be399dfe1fdf None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:10.766325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad417-f483-4d9e-8fd5-be399dfe1fdf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:10.766325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad417-f483-4d9e-8fd5-be399dfe1fdf None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:10.768183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 670/1352] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:27:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:10.773444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7be9c-ef4b-4269-9fba-967ea132f3fb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:10.773867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7be9c-ef4b-4269-9fba-967ea132f3fb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:10.774204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7be9c-ef4b-4269-9fba-967ea132f3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:10.859932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d0b7be9c-ef4b-4269-9fba-967ea132f3fb None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:10.861230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d0b7be9c-ef4b-4269-9fba-967ea132f3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:10.940433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:10.977734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 683/1353] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:12.376238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e96fc-e337-4601-b1ad-848560d6af1b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:12.376838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e96fc-e337-4601-b1ad-848560d6af1b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:12.376838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e96fc-e337-4601-b1ad-848560d6af1b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:12.378196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 671/1354] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:27:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:12.383177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c282070-6a85-4afb-a785-31c1ba76c0e7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:12.383589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c282070-6a85-4afb-a785-31c1ba76c0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:12.383765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c282070-6a85-4afb-a785-31c1ba76c0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:12.475000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6c282070-6a85-4afb-a785-31c1ba76c0e7 None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:12.476235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6c282070-6a85-4afb-a785-31c1ba76c0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:12.587359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 684/1355] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:27:16.093664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e913-dc81-40cf-a950-cf7e9ba778f3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:16.094574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e913-dc81-40cf-a950-cf7e9ba778f3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:16.095026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e913-dc81-40cf-a950-cf7e9ba778f3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:16.099271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 672/1356] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:16 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:16.102822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98b93c-2494-484a-ab36-eec2e04d4b33 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:16.103313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98b93c-2494-484a-ab36-eec2e04d4b33 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:16.103411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98b93c-2494-484a-ab36-eec2e04d4b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:16.164928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:16.197819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3b98b93c-2494-484a-ab36-eec2e04d4b33 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:16.199323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3b98b93c-2494-484a-ab36-eec2e04d4b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:16.312579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 685/1357] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:16.368742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7cc3-69ce-4bf5-9472-0e456c9dfd43 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:16.369316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7cc3-69ce-4bf5-9472-0e456c9dfd43 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:16.369800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7cc3-69ce-4bf5-9472-0e456c9dfd43 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:16.371567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 673/1358] 10.222.0.128 () {54 vars in 967 bytes} [Tue Jul 26 15:27:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:16.381405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef81f76-656f-4406-b6aa-cb68c759ee37 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:16.382007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef81f76-656f-4406-b6aa-cb68c759ee37 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:16.382531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef81f76-656f-4406-b6aa-cb68c759ee37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:16.479876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cef81f76-656f-4406-b6aa-cb68c759ee37 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:16.481577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cef81f76-656f-4406-b6aa-cb68c759ee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:16.536395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:16.613079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 686/1359] 10.222.0.128 () {58 vars in 1064 bytes} [Tue Jul 26 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:27:16.632139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7cc3-69ce-4bf5-9472-0e456c9dfd43 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:16.636354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f3d7cc3-69ce-4bf5-9472-0e456c9dfd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:16.793491 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7cc3-69ce-4bf5-9472-0e456c9dfd43 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:16.875019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:16.921161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437319a2-e5db-4b47-8c16-e9b02ec607cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:16.921745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437319a2-e5db-4b47-8c16-e9b02ec607cd service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:16.922158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437319a2-e5db-4b47-8c16-e9b02ec607cd service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:16.922580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437319a2-e5db-4b47-8c16-e9b02ec607cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:16.924230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-437319a2-e5db-4b47-8c16-e9b02ec607cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:17.024628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437319a2-e5db-4b47-8c16-e9b02ec607cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:17.026226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437319a2-e5db-4b47-8c16-e9b02ec607cd service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:17.046364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 674/1360] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:19.345938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b65da2-2586-46d9-80a5-c5f05d23157e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:19.346472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b65da2-2586-46d9-80a5-c5f05d23157e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:19.346593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b65da2-2586-46d9-80a5-c5f05d23157e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:19.347844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 687/1361] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:27:19.353579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3c46b-9ed1-4fb7-8b2e-557fbd2ba922 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:19.353994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3c46b-9ed1-4fb7-8b2e-557fbd2ba922 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:19.354180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3c46b-9ed1-4fb7-8b2e-557fbd2ba922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:19.425559 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bcc3c46b-9ed1-4fb7-8b2e-557fbd2ba922 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:19.426817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bcc3c46b-9ed1-4fb7-8b2e-557fbd2ba922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:19.475483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:19.555739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 675/1362] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:19.602363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b65da2-2586-46d9-80a5-c5f05d23157e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:19.603991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a8b65da2-2586-46d9-80a5-c5f05d23157e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:19.764344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b65da2-2586-46d9-80a5-c5f05d23157e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:19.775150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:19.889562 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448052f-340c-4ca8-9b62-48d18fdef7ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:19.890194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448052f-340c-4ca8-9b62-48d18fdef7ca service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:19.890474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448052f-340c-4ca8-9b62-48d18fdef7ca service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:19.890747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448052f-340c-4ca8-9b62-48d18fdef7ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:19.892412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2448052f-340c-4ca8-9b62-48d18fdef7ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:19.988391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448052f-340c-4ca8-9b62-48d18fdef7ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:19.990016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448052f-340c-4ca8-9b62-48d18fdef7ca service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:20.009139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 688/1363] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:22.552263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eab263-662c-423d-a334-e999fba9c75e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:22.552844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eab263-662c-423d-a334-e999fba9c75e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:22.552962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eab263-662c-423d-a334-e999fba9c75e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:22.554310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 676/1364] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:22.560514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643356d-8de6-4e43-a871-c9ad4078dc83 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:22.561149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643356d-8de6-4e43-a871-c9ad4078dc83 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:22.561149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643356d-8de6-4e43-a871-c9ad4078dc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:22.578292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:22.631753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b643356d-8de6-4e43-a871-c9ad4078dc83 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:22.632984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b643356d-8de6-4e43-a871-c9ad4078dc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:22.736895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 689/1365] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:22.784896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc367a73-56cd-4a56-b5f2-b652b11da809 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:22.785320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc367a73-56cd-4a56-b5f2-b652b11da809 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:22.785697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc367a73-56cd-4a56-b5f2-b652b11da809 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:22.787181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 677/1366] 10.222.0.128 () {54 vars in 967 bytes} [Tue Jul 26 15:27:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:22.792697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfe5fe-fc5e-4a15-87c1-fce407918cc9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:22.796571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfe5fe-fc5e-4a15-87c1-fce407918cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:22.796571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfe5fe-fc5e-4a15-87c1-fce407918cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:22.884772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-82cfe5fe-fc5e-4a15-87c1-fce407918cc9 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:22.886033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-82cfe5fe-fc5e-4a15-87c1-fce407918cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:22.928065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:23.018993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 690/1367] 10.222.0.128 () {58 vars in 1064 bytes} [Tue Jul 26 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:23.026756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc367a73-56cd-4a56-b5f2-b652b11da809 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:23.028881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dc367a73-56cd-4a56-b5f2-b652b11da809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:23.188661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc367a73-56cd-4a56-b5f2-b652b11da809 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:23.212611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:23.327467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a147c-da03-4d1a-8f87-59f08408453c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:23.328015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a147c-da03-4d1a-8f87-59f08408453c service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:23.328299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a147c-da03-4d1a-8f87-59f08408453c service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:23.328607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a147c-da03-4d1a-8f87-59f08408453c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:23.330289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bd3a147c-da03-4d1a-8f87-59f08408453c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:23.452179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a147c-da03-4d1a-8f87-59f08408453c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:23.452179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a147c-da03-4d1a-8f87-59f08408453c service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:23.473347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 678/1368] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:24.326596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02408c-7055-45ab-b8e2-b0ea5cc06e33 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:24.327184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02408c-7055-45ab-b8e2-b0ea5cc06e33 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:24.327184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02408c-7055-45ab-b8e2-b0ea5cc06e33 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:24.328699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 691/1369] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:24.333844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde3bbf-d183-4992-b6c4-f65a56d73513 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:24.334246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde3bbf-d183-4992-b6c4-f65a56d73513 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:24.334435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde3bbf-d183-4992-b6c4-f65a56d73513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:24.368293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:24.408776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3dde3bbf-d183-4992-b6c4-f65a56d73513 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:24.410983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3dde3bbf-d183-4992-b6c4-f65a56d73513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:24.518443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 679/1370] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:24.866507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637447d-1150-465e-aa75-7cd47e8ed80f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:24.866801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637447d-1150-465e-aa75-7cd47e8ed80f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:24.867080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637447d-1150-465e-aa75-7cd47e8ed80f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:24.868948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 692/1371] 10.222.0.128 () {54 vars in 969 bytes} [Tue Jul 26 15:27:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:24.874765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbee63-3604-46ff-97c1-a86b707447db None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:24.875041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbee63-3604-46ff-97c1-a86b707447db None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:24.875331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbee63-3604-46ff-97c1-a86b707447db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:24.966841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-addbee63-3604-46ff-97c1-a86b707447db None None] MFA Rules not processed for user `8ff6f527aa364534b2fdd0b770924e83`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:24.968300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-addbee63-3604-46ff-97c1-a86b707447db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:25.021257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:25.054525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 680/1372] 10.222.0.128 () {58 vars in 1066 bytes} [Tue Jul 26 15:27:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:25.063364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637447d-1150-465e-aa75-7cd47e8ed80f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:25.065094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8637447d-1150-465e-aa75-7cd47e8ed80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:25.179965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637447d-1150-465e-aa75-7cd47e8ed80f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:25.224135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:25.259866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1c958-7e30-436d-819d-8f87f0d054a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:25.261031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1c958-7e30-436d-819d-8f87f0d054a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:25.261520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1c958-7e30-436d-819d-8f87f0d054a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:25.262736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1c958-7e30-436d-819d-8f87f0d054a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:25.264723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-eac1c958-7e30-436d-819d-8f87f0d054a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:25.388479 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1c958-7e30-436d-819d-8f87f0d054a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:25.390014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1c958-7e30-436d-819d-8f87f0d054a4 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:25.410451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 693/1373] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:25.812268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318fdb6-6a81-42f6-9f09-b382a5998ecb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:25.812268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318fdb6-6a81-42f6-9f09-b382a5998ecb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:25.813147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318fdb6-6a81-42f6-9f09-b382a5998ecb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:25.813859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 681/1374] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:25.820514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91955df-1dcb-48cd-b479-48b57b5c862f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:25.820514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91955df-1dcb-48cd-b479-48b57b5c862f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:25.820514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91955df-1dcb-48cd-b479-48b57b5c862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:25.901776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c91955df-1dcb-48cd-b479-48b57b5c862f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:25.903775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c91955df-1dcb-48cd-b479-48b57b5c862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:25.924438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:26.004034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 694/1375] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:27:26.049704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318fdb6-6a81-42f6-9f09-b382a5998ecb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:26.051411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9318fdb6-6a81-42f6-9f09-b382a5998ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:26.202354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318fdb6-6a81-42f6-9f09-b382a5998ecb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:26.255361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:26.318147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1c67b-2af8-4974-93e2-528734fefc75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:26.318776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1c67b-2af8-4974-93e2-528734fefc75 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:26.319065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1c67b-2af8-4974-93e2-528734fefc75 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:26.319347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1c67b-2af8-4974-93e2-528734fefc75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:26.321020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bbf1c67b-2af8-4974-93e2-528734fefc75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:26.412028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1c67b-2af8-4974-93e2-528734fefc75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:26.413673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1c67b-2af8-4974-93e2-528734fefc75 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:26.432315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 682/1376] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:28.631371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641480c-4bac-4100-9384-bae01f698ac8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:28.631891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641480c-4bac-4100-9384-bae01f698ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:28.631998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641480c-4bac-4100-9384-bae01f698ac8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:28.633635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 695/1377] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:28.640058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff9d63-b5b1-41b2-8d18-947385689430 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:28.640601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff9d63-b5b1-41b2-8d18-947385689430 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:28.640701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff9d63-b5b1-41b2-8d18-947385689430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:28.708309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-02ff9d63-b5b1-41b2-8d18-947385689430 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:28.709453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-02ff9d63-b5b1-41b2-8d18-947385689430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:28.716213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:28.807141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 683/1378] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:28.851543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641480c-4bac-4100-9384-bae01f698ac8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:28.853549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9641480c-4bac-4100-9384-bae01f698ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:29.016628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641480c-4bac-4100-9384-bae01f698ac8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:29.020665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:29.162893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032b3e6-25fb-4355-8d70-61b7c25c4a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:29.163894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b3e6-25fb-4355-8d70-61b7c25c4a90 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:29.164189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b3e6-25fb-4355-8d70-61b7c25c4a90 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:29.164489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b3e6-25fb-4355-8d70-61b7c25c4a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:29.166166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1032b3e6-25fb-4355-8d70-61b7c25c4a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:29.278874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032b3e6-25fb-4355-8d70-61b7c25c4a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:29.280661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032b3e6-25fb-4355-8d70-61b7c25c4a90 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:29.301314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 696/1379] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:31.540456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939fe14-7ef2-4867-a94f-9d9cfd4a9a00 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:31.541034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939fe14-7ef2-4867-a94f-9d9cfd4a9a00 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:31.541034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939fe14-7ef2-4867-a94f-9d9cfd4a9a00 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:31.544167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 684/1380] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:31 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:31.548319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea33f-d742-4ab4-a690-f502c465b09b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:31.548826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea33f-d742-4ab4-a690-f502c465b09b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:31.548928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea33f-d742-4ab4-a690-f502c465b09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:31.562605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:31.629341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fc3ea33f-d742-4ab4-a690-f502c465b09b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:31.630682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fc3ea33f-d742-4ab4-a690-f502c465b09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:31.739580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 697/1381] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:31.789767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939fe14-7ef2-4867-a94f-9d9cfd4a9a00 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:31.791429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f939fe14-7ef2-4867-a94f-9d9cfd4a9a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:31.945471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:31.956995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939fe14-7ef2-4867-a94f-9d9cfd4a9a00 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:32.097160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39d497-21f6-472c-93db-c528ce0ed374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:32.097866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39d497-21f6-472c-93db-c528ce0ed374 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:32.098262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39d497-21f6-472c-93db-c528ce0ed374 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:32.098628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39d497-21f6-472c-93db-c528ce0ed374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:32.100383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6d39d497-21f6-472c-93db-c528ce0ed374 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:32.209419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39d497-21f6-472c-93db-c528ce0ed374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:32.211169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39d497-21f6-472c-93db-c528ce0ed374 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:32.225179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:32.237139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 685/1382] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:34.495322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37208f4d-5b59-4052-a459-13e17fd48c3f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:34.496160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37208f4d-5b59-4052-a459-13e17fd48c3f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:34.496160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37208f4d-5b59-4052-a459-13e17fd48c3f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:34.497326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 698/1383] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:27:34.502552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195e83-b8d2-4049-a6bf-ce3f171d7971 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:34.503104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195e83-b8d2-4049-a6bf-ce3f171d7971 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:34.503104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195e83-b8d2-4049-a6bf-ce3f171d7971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:34.573096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-88195e83-b8d2-4049-a6bf-ce3f171d7971 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:34.574552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-88195e83-b8d2-4049-a6bf-ce3f171d7971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:34.679799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 686/1384] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:34.723816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37208f4d-5b59-4052-a459-13e17fd48c3f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:34.725372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-37208f4d-5b59-4052-a459-13e17fd48c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:34.815635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:34.886159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37208f4d-5b59-4052-a459-13e17fd48c3f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:35.017196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe113c0-9821-4b3b-892b-86b6aa6abdae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:35.017743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe113c0-9821-4b3b-892b-86b6aa6abdae service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:35.018025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe113c0-9821-4b3b-892b-86b6aa6abdae service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:35.018300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe113c0-9821-4b3b-892b-86b6aa6abdae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:35.019957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9fe113c0-9821-4b3b-892b-86b6aa6abdae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:35.071818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:35.114471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe113c0-9821-4b3b-892b-86b6aa6abdae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:35.116172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe113c0-9821-4b3b-892b-86b6aa6abdae service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:35.136187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 699/1385] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:37.416948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9ab57-4fd5-4ead-988a-9468ff6cb731 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:37.417622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9ab57-4fd5-4ead-988a-9468ff6cb731 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:37.417622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9ab57-4fd5-4ead-988a-9468ff6cb731 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:37.419514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 687/1386] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:37.425228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb4ed-0d07-4ff9-8a4e-73f207845d49 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:37.425696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb4ed-0d07-4ff9-8a4e-73f207845d49 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:37.425860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb4ed-0d07-4ff9-8a4e-73f207845d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:37.497574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-186eb4ed-0d07-4ff9-8a4e-73f207845d49 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:37.498866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-186eb4ed-0d07-4ff9-8a4e-73f207845d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:37.603126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 700/1387] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:37.649057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9ab57-4fd5-4ead-988a-9468ff6cb731 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:37.650582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bef9ab57-4fd5-4ead-988a-9468ff6cb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:37.744760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:37.821365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9ab57-4fd5-4ead-988a-9468ff6cb731 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:37.945634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bafc4-92e7-489d-b77f-2d4fd0b33f35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:37.946263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bafc4-92e7-489d-b77f-2d4fd0b33f35 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:37.946551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bafc4-92e7-489d-b77f-2d4fd0b33f35 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:37.946830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bafc4-92e7-489d-b77f-2d4fd0b33f35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:37.948551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ad4bafc4-92e7-489d-b77f-2d4fd0b33f35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:37.991152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:38.050519 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bafc4-92e7-489d-b77f-2d4fd0b33f35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:38.052144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bafc4-92e7-489d-b77f-2d4fd0b33f35 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:38.071753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 688/1388] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:40.505639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549818a-e28f-4277-928d-ff734101b219 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:40.506206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549818a-e28f-4277-928d-ff734101b219 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:40.506206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549818a-e28f-4277-928d-ff734101b219 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:40.507603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 701/1389] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:40.513802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a19e0-4a90-4fe1-832c-2a0a3ed6822f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:40.514257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a19e0-4a90-4fe1-832c-2a0a3ed6822f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:40.514374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a19e0-4a90-4fe1-832c-2a0a3ed6822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:40.584389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d46a19e0-4a90-4fe1-832c-2a0a3ed6822f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:40.585629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d46a19e0-4a90-4fe1-832c-2a0a3ed6822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:40.682137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 689/1390] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:27:40.727661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549818a-e28f-4277-928d-ff734101b219 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:40.729217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2549818a-e28f-4277-928d-ff734101b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:40.762404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:40.882261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549818a-e28f-4277-928d-ff734101b219 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:40.991058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:41.002534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f1f6d-a86e-4329-9b04-30e5f0b81127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:41.003167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f1f6d-a86e-4329-9b04-30e5f0b81127 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:41.003450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f1f6d-a86e-4329-9b04-30e5f0b81127 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:41.003724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f1f6d-a86e-4329-9b04-30e5f0b81127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:41.005420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b53f1f6d-a86e-4329-9b04-30e5f0b81127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:41.007760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ef656-a2f8-499d-a3ab-572f234f4ab3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:41.008052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ef656-a2f8-499d-a3ab-572f234f4ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:41.008340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ef656-a2f8-499d-a3ab-572f234f4ab3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:41.009725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 690/1391] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:27:41.015440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021551-96e4-4786-9c91-f562536e152c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:41.015728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021551-96e4-4786-9c91-f562536e152c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:41.016007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021551-96e4-4786-9c91-f562536e152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:41.032517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:41.085328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-34021551-96e4-4786-9c91-f562536e152c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:41.086551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-34021551-96e4-4786-9c91-f562536e152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:41.100766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f1f6d-a86e-4329-9b04-30e5f0b81127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:41.102678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f1f6d-a86e-4329-9b04-30e5f0b81127 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:41.120488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 702/1392] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:41.189309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 691/1393] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:27:41.248121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deca3c-2f6a-46fb-96ff-1d178ff8cf9b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:41.248121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deca3c-2f6a-46fb-96ff-1d178ff8cf9b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:41.248121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deca3c-2f6a-46fb-96ff-1d178ff8cf9b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:41.249506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 703/1394] 10.222.0.128 () {54 vars in 971 bytes} [Tue Jul 26 15:27:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:41.255019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd44e-efd0-43dd-8bb8-ce54015989d4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:41.255310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd44e-efd0-43dd-8bb8-ce54015989d4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:41.255589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd44e-efd0-43dd-8bb8-ce54015989d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:41.341663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-22cfd44e-efd0-43dd-8bb8-ce54015989d4 None None] MFA Rules not processed for user `0207287721f5489f8ac7a258356e57bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:41.342920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-22cfd44e-efd0-43dd-8bb8-ce54015989d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:41.384574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:41.432882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 692/1395] 10.222.0.128 () {58 vars in 1068 bytes} [Tue Jul 26 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:41.441652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deca3c-2f6a-46fb-96ff-1d178ff8cf9b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:41.443430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-32deca3c-2f6a-46fb-96ff-1d178ff8cf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:41.558438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deca3c-2f6a-46fb-96ff-1d178ff8cf9b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:41.572189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:41.635265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bd0bb-2b47-4732-ba9d-c0a304361659 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:41.635889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bd0bb-2b47-4732-ba9d-c0a304361659 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:41.636189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bd0bb-2b47-4732-ba9d-c0a304361659 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:41.636484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bd0bb-2b47-4732-ba9d-c0a304361659 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:41.638183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0b6bd0bb-2b47-4732-ba9d-c0a304361659 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:41.756377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bd0bb-2b47-4732-ba9d-c0a304361659 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:41.758002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bd0bb-2b47-4732-ba9d-c0a304361659 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:41.776707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 704/1396] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:42.679230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38a6f1-acf9-4760-8f1b-c5d3969ae87d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:42.679717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38a6f1-acf9-4760-8f1b-c5d3969ae87d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:42.679811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38a6f1-acf9-4760-8f1b-c5d3969ae87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:42.749940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ee38a6f1-acf9-4760-8f1b-c5d3969ae87d None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:42.751130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee38a6f1-acf9-4760-8f1b-c5d3969ae87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:42.843359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 693/1397] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:27:42.891354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551198f-25e1-4c00-92bf-a4bbc099c522 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:42.891852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551198f-25e1-4c00-92bf-a4bbc099c522 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:42.891949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551198f-25e1-4c00-92bf-a4bbc099c522 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:42.893294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 705/1398] 10.222.0.128 () {54 vars in 948 bytes} [Tue Jul 26 15:27:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:42.899514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cf25-0f36-49ec-bfa5-019291973300 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:42.899807 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cf25-0f36-49ec-bfa5-019291973300 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:42.900083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cf25-0f36-49ec-bfa5-019291973300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:42.963234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:42.984413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-caf5cf25-0f36-49ec-bfa5-019291973300 None None] MFA Rules not processed for user `97295dc4a04143fe879bef062964e4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:42.986320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-caf5cf25-0f36-49ec-bfa5-019291973300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:43.101053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 694/1399] 10.222.0.128 () {58 vars in 1045 bytes} [Tue Jul 26 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:43.112869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551198f-25e1-4c00-92bf-a4bbc099c522 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:43.114455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d551198f-25e1-4c00-92bf-a4bbc099c522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:43.162572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:43.260616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551198f-25e1-4c00-92bf-a4bbc099c522 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:43.341857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf5991-64c5-499c-8365-ebba304cf8f1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:43.342154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf5991-64c5-499c-8365-ebba304cf8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:43.342432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf5991-64c5-499c-8365-ebba304cf8f1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:43.343778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 695/1400] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:27:43.348805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f243f3-8527-4fa1-9f78-9e0f0c46c859 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:43.349259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f243f3-8527-4fa1-9f78-9e0f0c46c859 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:43.349540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f243f3-8527-4fa1-9f78-9e0f0c46c859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:43.369255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:43.369876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:43.370163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:43.370475 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:43.373089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:43.403397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:43.418326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d4f243f3-8527-4fa1-9f78-9e0f0c46c859 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:43.419551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d4f243f3-8527-4fa1-9f78-9e0f0c46c859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:43.461523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:43.463298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:43.472137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 706/1401] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:27:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:27:43.483628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:43.529535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 696/1402] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:27:43.576142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:43.577386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:43.669082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ef1de-5a96-4f98-9d1e-827bc4143cfc service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:43.742886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:43.787568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03235c-8970-4a5d-ab09-ad345ae63b28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:43.788118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03235c-8970-4a5d-ab09-ad345ae63b28 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:43.788402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03235c-8970-4a5d-ab09-ad345ae63b28 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:43.788717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03235c-8970-4a5d-ab09-ad345ae63b28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:43.790373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ab03235c-8970-4a5d-ab09-ad345ae63b28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:43.893936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03235c-8970-4a5d-ab09-ad345ae63b28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:43.895543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03235c-8970-4a5d-ab09-ad345ae63b28 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:43.919746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 707/1403] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:46.152408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88464-79e7-4392-8feb-0201736b12e4 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:46.152982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88464-79e7-4392-8feb-0201736b12e4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:46.152982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88464-79e7-4392-8feb-0201736b12e4 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:46.154356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 697/1404] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:46.160257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72262-2df3-4c46-a2e1-645e02ee606e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:46.160829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72262-2df3-4c46-a2e1-645e02ee606e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:46.161040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72262-2df3-4c46-a2e1-645e02ee606e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:46.244234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0fb72262-2df3-4c46-a2e1-645e02ee606e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:46.245655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0fb72262-2df3-4c46-a2e1-645e02ee606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:46.292649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:46.376585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 708/1405] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:46.422812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88464-79e7-4392-8feb-0201736b12e4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:46.424702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-95a88464-79e7-4392-8feb-0201736b12e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:46.604095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88464-79e7-4392-8feb-0201736b12e4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:46.655007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:46.739113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5da88-a210-4ea5-a44b-eb6224574b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:46.739731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5da88-a210-4ea5-a44b-eb6224574b38 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:46.740010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5da88-a210-4ea5-a44b-eb6224574b38 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:46.740282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5da88-a210-4ea5-a44b-eb6224574b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:46.741958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63d5da88-a210-4ea5-a44b-eb6224574b38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:46.843919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5da88-a210-4ea5-a44b-eb6224574b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:46.845683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5da88-a210-4ea5-a44b-eb6224574b38 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:46.866213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 698/1406] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:49.081647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6e00-d402-4724-90de-e43e071997aa None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:49.082205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6e00-d402-4724-90de-e43e071997aa None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:49.082205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6e00-d402-4724-90de-e43e071997aa None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:49.083620 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 709/1407] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:27:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:27:49.089093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e854e-85ac-4a0a-8c01-bc02f24d4e49 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:49.090449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e854e-85ac-4a0a-8c01-bc02f24d4e49 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:49.090735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e854e-85ac-4a0a-8c01-bc02f24d4e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:49.156561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:49.164738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-185e854e-85ac-4a0a-8c01-bc02f24d4e49 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:27:49.169805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-185e854e-85ac-4a0a-8c01-bc02f24d4e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:49.286532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 699/1408] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:27:49.335071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf6e00-d402-4724-90de-e43e071997aa None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:27:49.336941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-03cf6e00-d402-4724-90de-e43e071997aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:49.512263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf6e00-d402-4724-90de-e43e071997aa None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:27:49.534863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:49.649733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc45d3-3e02-4bf4-8448-387b448430d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:27:49.650267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc45d3-3e02-4bf4-8448-387b448430d0 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:27:49.650552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc45d3-3e02-4bf4-8448-387b448430d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:27:49.650831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc45d3-3e02-4bf4-8448-387b448430d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:27:49.653192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d5bc45d3-3e02-4bf4-8448-387b448430d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:27:49.754981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc45d3-3e02-4bf4-8448-387b448430d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:27:49.756676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc45d3-3e02-4bf4-8448-387b448430d0 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:27:49.780094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 710/1409] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:27:59.284430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=44 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:27:59.773629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:07.430089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e59f2-fd9f-44d9-864f-9140f916016a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:07.430634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e59f2-fd9f-44d9-864f-9140f916016a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:07.430634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e59f2-fd9f-44d9-864f-9140f916016a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:07.432647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 700/1410] 10.222.0.128 () {54 vars in 923 bytes} [Tue Jul 26 15:28:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:07.438225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6f30-2a7c-4335-a56f-e1bf3e43ceaa None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:07.438726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6f30-2a7c-4335-a56f-e1bf3e43ceaa None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:07.438833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6f30-2a7c-4335-a56f-e1bf3e43ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:07.517439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4a8e6f30-2a7c-4335-a56f-e1bf3e43ceaa None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:07.518676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4a8e6f30-2a7c-4335-a56f-e1bf3e43ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:07.626248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 711/1411] 10.222.0.128 () {58 vars in 1020 bytes} [Tue Jul 26 15:28:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:07.641514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cc145-9b68-4ff0-9d46-5b3b90b1fd9d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:07.641800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cc145-9b68-4ff0-9d46-5b3b90b1fd9d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:07.642082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cc145-9b68-4ff0-9d46-5b3b90b1fd9d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:07.643414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 701/1412] 10.222.0.128 () {54 vars in 923 bytes} [Tue Jul 26 15:28:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:07.648090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dbfee-4b4c-4396-ba82-a4a95f852843 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:07.648374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dbfee-4b4c-4396-ba82-a4a95f852843 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:07.648702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dbfee-4b4c-4396-ba82-a4a95f852843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:07.721312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:07.724354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-494dbfee-4b4c-4396-ba82-a4a95f852843 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:07.725856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-494dbfee-4b4c-4396-ba82-a4a95f852843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:07.842494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 712/1413] 10.222.0.128 () {58 vars in 1020 bytes} [Tue Jul 26 15:28:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:28:10.321250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d447-2da0-487c-9555-62462a767d48 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:10.321796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d447-2da0-487c-9555-62462a767d48 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:10.321796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d447-2da0-487c-9555-62462a767d48 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:10.323380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:10.328571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ecdb6-dc5a-471d-82f4-6e640fe1eac8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:10.328990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ecdb6-dc5a-471d-82f4-6e640fe1eac8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:10.329135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ecdb6-dc5a-471d-82f4-6e640fe1eac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:10.395194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0f5ecdb6-dc5a-471d-82f4-6e640fe1eac8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:10.396413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0f5ecdb6-dc5a-471d-82f4-6e640fe1eac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:10.452890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:10.498412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:10.541773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5d447-2da0-487c-9555-62462a767d48 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:10.543316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1b5d447-2da0-487c-9555-62462a767d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:10.688740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5d447-2da0-487c-9555-62462a767d48 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:10.761423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:10.803445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1cb4-f4b0-4929-b32b-4488e3907ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:10.804100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1cb4-f4b0-4929-b32b-4488e3907ccc service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:10.804385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1cb4-f4b0-4929-b32b-4488e3907ccc service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:10.804696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1cb4-f4b0-4929-b32b-4488e3907ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:10.806345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9a4c1cb4-f4b0-4929-b32b-4488e3907ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:10.895418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1cb4-f4b0-4929-b32b-4488e3907ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:10.897077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1cb4-f4b0-4929-b32b-4488e3907ccc service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:10.914666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 703/1416] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:11.827327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b47d-c7b0-4533-a4a6-93b4c33bbe65 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:11.827853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b47d-c7b0-4533-a4a6-93b4c33bbe65 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:11.827853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b47d-c7b0-4533-a4a6-93b4c33bbe65 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:11.829245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:11.834403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272afac4-2851-41c1-806d-939f3bbb083a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:11.834845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272afac4-2851-41c1-806d-939f3bbb083a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:11.834968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272afac4-2851-41c1-806d-939f3bbb083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:11.912512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-272afac4-2851-41c1-806d-939f3bbb083a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:11.913732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-272afac4-2851-41c1-806d-939f3bbb083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:11.940756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:12.009726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:12.166112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b47d-c7b0-4533-a4a6-93b4c33bbe65 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:12.167959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7cf6b47d-c7b0-4533-a4a6-93b4c33bbe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:12.314480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b47d-c7b0-4533-a4a6-93b4c33bbe65 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:12.357203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:12.430415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8f15-b6a2-4204-8700-841b81f4e417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:12.430957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8f15-b6a2-4204-8700-841b81f4e417 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:12.431241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8f15-b6a2-4204-8700-841b81f4e417 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:12.431561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8f15-b6a2-4204-8700-841b81f4e417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:12.433236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-932a8f15-b6a2-4204-8700-841b81f4e417 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:12.527774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8f15-b6a2-4204-8700-841b81f4e417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:12.529410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8f15-b6a2-4204-8700-841b81f4e417 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:12.547723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 715/1419] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:13.455214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770fe3-7dcc-4c42-9fa3-f38a84355374 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:13.455776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770fe3-7dcc-4c42-9fa3-f38a84355374 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:13.455776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770fe3-7dcc-4c42-9fa3-f38a84355374 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:13.457219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:13.462314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de120a-e417-4d8b-87b5-bcd58d18765b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:13.462748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de120a-e417-4d8b-87b5-bcd58d18765b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:13.463029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de120a-e417-4d8b-87b5-bcd58d18765b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:13.519333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:13.541329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-06de120a-e417-4d8b-87b5-bcd58d18765b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:13.542647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-06de120a-e417-4d8b-87b5-bcd58d18765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:13.639942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:13.796023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe770fe3-7dcc-4c42-9fa3-f38a84355374 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:13.797579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fe770fe3-7dcc-4c42-9fa3-f38a84355374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:13.945294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe770fe3-7dcc-4c42-9fa3-f38a84355374 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:13.957810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:14.061781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c554d-14fa-4a49-9c17-febd6e00977e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:14.062320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c554d-14fa-4a49-9c17-febd6e00977e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:14.062598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c554d-14fa-4a49-9c17-febd6e00977e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:14.062866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c554d-14fa-4a49-9c17-febd6e00977e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:14.064516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3a5c554d-14fa-4a49-9c17-febd6e00977e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:14.157582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c554d-14fa-4a49-9c17-febd6e00977e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:14.159189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c554d-14fa-4a49-9c17-febd6e00977e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:14.178475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 706/1422] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:15.054503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b122d-e3d4-4782-9b33-840bcbe0d143 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:15.055056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b122d-e3d4-4782-9b33-840bcbe0d143 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:15.055056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b122d-e3d4-4782-9b33-840bcbe0d143 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:15.056998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:15.062314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54fae-a8f7-4a4f-9c95-94bc96628d7c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:15.062763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54fae-a8f7-4a4f-9c95-94bc96628d7c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:15.062883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54fae-a8f7-4a4f-9c95-94bc96628d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:15.082704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:15.143211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-23c54fae-a8f7-4a4f-9c95-94bc96628d7c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:15.144434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-23c54fae-a8f7-4a4f-9c95-94bc96628d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:15.249220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:15.402272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b122d-e3d4-4782-9b33-840bcbe0d143 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:15.403813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bd2b122d-e3d4-4782-9b33-840bcbe0d143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:15.516184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:15.554344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b122d-e3d4-4782-9b33-840bcbe0d143 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:15.671744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a342e-28e3-47a6-ace8-9f30052418a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:15.672391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a342e-28e3-47a6-ace8-9f30052418a3 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:15.672710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a342e-28e3-47a6-ace8-9f30052418a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:15.672989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a342e-28e3-47a6-ace8-9f30052418a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:15.674669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-847a342e-28e3-47a6-ace8-9f30052418a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:15.756343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:15.771071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a342e-28e3-47a6-ace8-9f30052418a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:15.772649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a342e-28e3-47a6-ace8-9f30052418a3 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:15.791365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 718/1425] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:16.659043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41ca46-08b7-4048-9872-651ab0452c3a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:16.664835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41ca46-08b7-4048-9872-651ab0452c3a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:16.664835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41ca46-08b7-4048-9872-651ab0452c3a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:16.664835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:16.666091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9712b59-1326-4f2b-9c29-295ca12e20c8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:16.666541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9712b59-1326-4f2b-9c29-295ca12e20c8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:16.666825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9712b59-1326-4f2b-9c29-295ca12e20c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:16.865253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b9712b59-1326-4f2b-9c29-295ca12e20c8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:16.866398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b9712b59-1326-4f2b-9c29-295ca12e20c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:16.964226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:28:17.114649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41ca46-08b7-4048-9872-651ab0452c3a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:17.116227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af41ca46-08b7-4048-9872-651ab0452c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:17.202588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:17.266742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41ca46-08b7-4048-9872-651ab0452c3a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:17.381290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57b9f2-8a33-4b0e-9931-06d8b80048e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:17.381829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57b9f2-8a33-4b0e-9931-06d8b80048e4 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:17.382112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57b9f2-8a33-4b0e-9931-06d8b80048e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:17.382390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57b9f2-8a33-4b0e-9931-06d8b80048e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:17.384030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8c57b9f2-8a33-4b0e-9931-06d8b80048e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:17.432414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:17.474449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57b9f2-8a33-4b0e-9931-06d8b80048e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:17.476318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57b9f2-8a33-4b0e-9931-06d8b80048e4 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:17.494706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 709/1428] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:18.398361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3f7a8-1fa5-40dd-98fe-ff52e8333374 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:18.398947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3f7a8-1fa5-40dd-98fe-ff52e8333374 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:18.398947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3f7a8-1fa5-40dd-98fe-ff52e8333374 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:18.400959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:18.406237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d32234-2dfa-4645-86cd-fe1855e1cd2d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:18.406714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d32234-2dfa-4645-86cd-fe1855e1cd2d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:18.406821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d32234-2dfa-4645-86cd-fe1855e1cd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:18.483531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-78d32234-2dfa-4645-86cd-fe1855e1cd2d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:18.484782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-78d32234-2dfa-4645-86cd-fe1855e1cd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:18.582043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:18.735616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3f7a8-1fa5-40dd-98fe-ff52e8333374 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:18.737160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ded3f7a8-1fa5-40dd-98fe-ff52e8333374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:18.786789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:18.885574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3f7a8-1fa5-40dd-98fe-ff52e8333374 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:18.996606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ebece-f751-4c9c-b999-5b8b18355609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:18.997219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ebece-f751-4c9c-b999-5b8b18355609 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:18.997495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ebece-f751-4c9c-b999-5b8b18355609 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:18.997763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ebece-f751-4c9c-b999-5b8b18355609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:18.999370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0a0ebece-f751-4c9c-b999-5b8b18355609 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:19.006216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:19.088409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ebece-f751-4c9c-b999-5b8b18355609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:19.090056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ebece-f751-4c9c-b999-5b8b18355609 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:19.108956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 721/1431] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:19.969798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54e79-1c96-428e-a840-381d3416cfa7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:19.970358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54e79-1c96-428e-a840-381d3416cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:19.970358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54e79-1c96-428e-a840-381d3416cfa7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:19.971693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:19.976793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73c13-6888-4a08-8fd6-644bd8141097 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:19.977196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73c13-6888-4a08-8fd6-644bd8141097 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:19.977354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73c13-6888-4a08-8fd6-644bd8141097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:20.051187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-83a73c13-6888-4a08-8fd6-644bd8141097 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:20.052383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-83a73c13-6888-4a08-8fd6-644bd8141097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:20.119303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:20.147934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:20.301665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54e79-1c96-428e-a840-381d3416cfa7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:20.303095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fbb54e79-1c96-428e-a840-381d3416cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:20.321300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:20.453800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54e79-1c96-428e-a840-381d3416cfa7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:20.544190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:20.570731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a343d-9dfa-4d3c-8e90-906a785718a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:20.571226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a343d-9dfa-4d3c-8e90-906a785718a8 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:20.571714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a343d-9dfa-4d3c-8e90-906a785718a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:20.572332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a343d-9dfa-4d3c-8e90-906a785718a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:20.574351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d9a343d-9dfa-4d3c-8e90-906a785718a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:20.662476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a343d-9dfa-4d3c-8e90-906a785718a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:20.664084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a343d-9dfa-4d3c-8e90-906a785718a8 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:20.683341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 712/1434] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:21.559896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ce64b-a64e-4bd4-a7b4-7a45d344b97e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:21.560490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ce64b-a64e-4bd4-a7b4-7a45d344b97e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:21.560490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ce64b-a64e-4bd4-a7b4-7a45d344b97e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:21.561765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:28:21.566938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4402d-064f-46e3-9020-13bb3836006d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:21.567388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4402d-064f-46e3-9020-13bb3836006d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:21.567483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4402d-064f-46e3-9020-13bb3836006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:21.645276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-19f4402d-064f-46e3-9020-13bb3836006d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:21.646509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-19f4402d-064f-46e3-9020-13bb3836006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:21.687533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:21.744299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:21.896257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ce64b-a64e-4bd4-a7b4-7a45d344b97e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:21.897717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-eb7ce64b-a64e-4bd4-a7b4-7a45d344b97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:22.044457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ce64b-a64e-4bd4-a7b4-7a45d344b97e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:22.098266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:22.159030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24df3f-0cb1-421a-b1c3-273ed96d961d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:22.159553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24df3f-0cb1-421a-b1c3-273ed96d961d service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:22.159837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24df3f-0cb1-421a-b1c3-273ed96d961d service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:22.160122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24df3f-0cb1-421a-b1c3-273ed96d961d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:22.161791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2a24df3f-0cb1-421a-b1c3-273ed96d961d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:22.264437 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24df3f-0cb1-421a-b1c3-273ed96d961d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:22.267383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24df3f-0cb1-421a-b1c3-273ed96d961d service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:22.283922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 724/1437] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:23.178376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739ca3a-0a93-41ce-be79-a6748c906c5b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:23.178977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739ca3a-0a93-41ce-be79-a6748c906c5b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:23.178977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739ca3a-0a93-41ce-be79-a6748c906c5b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:23.180357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:28:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:23.185894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931737b-7226-4f08-91d4-4cd81a3f0e69 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:23.186329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931737b-7226-4f08-91d4-4cd81a3f0e69 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:23.186435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931737b-7226-4f08-91d4-4cd81a3f0e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:23.263432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c931737b-7226-4f08-91d4-4cd81a3f0e69 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:23.264676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c931737b-7226-4f08-91d4-4cd81a3f0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:23.268241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:23.363366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:23.514570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739ca3a-0a93-41ce-be79-a6748c906c5b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:23.516000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4739ca3a-0a93-41ce-be79-a6748c906c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:23.666469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739ca3a-0a93-41ce-be79-a6748c906c5b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:23.693670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:23.781620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd17be-1622-40f9-b044-9e5e41efa68f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:23.782150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd17be-1622-40f9-b044-9e5e41efa68f service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:23.782433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd17be-1622-40f9-b044-9e5e41efa68f service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:23.782709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd17be-1622-40f9-b044-9e5e41efa68f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:23.784334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c7dd17be-1622-40f9-b044-9e5e41efa68f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:23.874751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd17be-1622-40f9-b044-9e5e41efa68f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:23.876374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd17be-1622-40f9-b044-9e5e41efa68f service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:23.894213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 715/1440] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:25.812103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f530-56cb-4061-8791-26af87352fe6 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:25.812674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f530-56cb-4061-8791-26af87352fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:25.812674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f530-56cb-4061-8791-26af87352fe6 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:25.814046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:25.819730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853329-0986-49b0-b639-6e1bb49fe2c7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:25.820185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853329-0986-49b0-b639-6e1bb49fe2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:25.820425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853329-0986-49b0-b639-6e1bb49fe2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:25.865002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:25.893619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5c853329-0986-49b0-b639-6e1bb49fe2c7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:25.894847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5c853329-0986-49b0-b639-6e1bb49fe2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:25.994824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:26.034905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f530-56cb-4061-8791-26af87352fe6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:26.036485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1120f530-56cb-4061-8791-26af87352fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:26.140919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f530-56cb-4061-8791-26af87352fe6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:26.173073 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:26.215690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544334b-6862-44ed-8c8d-e861b332e64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:26.216643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544334b-6862-44ed-8c8d-e861b332e64b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:26.217170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544334b-6862-44ed-8c8d-e861b332e64b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:26.217865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544334b-6862-44ed-8c8d-e861b332e64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:26.219612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3544334b-6862-44ed-8c8d-e861b332e64b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:26.334405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544334b-6862-44ed-8c8d-e861b332e64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:26.335937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544334b-6862-44ed-8c8d-e861b332e64b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:26.354787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 727/1443] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:28.556833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95af9f-fa2c-46f1-aca6-6a7c9457ce3c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:28.557360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95af9f-fa2c-46f1-aca6-6a7c9457ce3c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:28.557360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95af9f-fa2c-46f1-aca6-6a7c9457ce3c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:28.558716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:28.564097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b73a-6bce-4385-96e9-22c9ce7f948b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:28.564553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b73a-6bce-4385-96e9-22c9ce7f948b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:28.564668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b73a-6bce-4385-96e9-22c9ce7f948b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:28.631115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dc83b73a-6bce-4385-96e9-22c9ce7f948b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:28.632339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dc83b73a-6bce-4385-96e9-22c9ce7f948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:28.645604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:28.733350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:28.771535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95af9f-fa2c-46f1-aca6-6a7c9457ce3c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:28.773129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dd95af9f-fa2c-46f1-aca6-6a7c9457ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:28.878717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95af9f-fa2c-46f1-aca6-6a7c9457ce3c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:28.891971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:28.955431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f331c-a5ef-493d-9d6d-de588196e19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:28.956052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f331c-a5ef-493d-9d6d-de588196e19a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:28.956331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f331c-a5ef-493d-9d6d-de588196e19a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:28.957171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f331c-a5ef-493d-9d6d-de588196e19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:28.958252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-468f331c-a5ef-493d-9d6d-de588196e19a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:29.070801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f331c-a5ef-493d-9d6d-de588196e19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:29.072296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f331c-a5ef-493d-9d6d-de588196e19a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:29.090647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 718/1446] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:31.468789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08992a94-5e8e-4e1f-8542-771f27e0fbc3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:31.469204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08992a94-5e8e-4e1f-8542-771f27e0fbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:31.469342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08992a94-5e8e-4e1f-8542-771f27e0fbc3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:31.470705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:31.476176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbd858-f9cf-4ccf-8d97-ebf531cadf0d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:31.476636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbd858-f9cf-4ccf-8d97-ebf531cadf0d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:31.476775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbd858-f9cf-4ccf-8d97-ebf531cadf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:31.525003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:31.542847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-70dbd858-f9cf-4ccf-8d97-ebf531cadf0d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:31.544071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-70dbd858-f9cf-4ccf-8d97-ebf531cadf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:31.642446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:31.681267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08992a94-5e8e-4e1f-8542-771f27e0fbc3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:31.683393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-08992a94-5e8e-4e1f-8542-771f27e0fbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:31.785964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08992a94-5e8e-4e1f-8542-771f27e0fbc3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:31.825955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:31.860309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad3c38-6b7a-4524-8bc9-3a89a5ace481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:31.860957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3c38-6b7a-4524-8bc9-3a89a5ace481 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:31.861239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3c38-6b7a-4524-8bc9-3a89a5ace481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:31.861513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3c38-6b7a-4524-8bc9-3a89a5ace481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:31.863128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e9ad3c38-6b7a-4524-8bc9-3a89a5ace481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:31.971439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad3c38-6b7a-4524-8bc9-3a89a5ace481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:31.972923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad3c38-6b7a-4524-8bc9-3a89a5ace481 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:31.992298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 730/1449] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:34.166606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889090c-2e20-4262-98fe-b03f2a57e5f7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:34.167176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889090c-2e20-4262-98fe-b03f2a57e5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:34.167176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889090c-2e20-4262-98fe-b03f2a57e5f7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:34.168740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:34.173929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9d7e6-2ea3-4088-9cdf-e80b0bf2e4a8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:34.174404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9d7e6-2ea3-4088-9cdf-e80b0bf2e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:34.174504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9d7e6-2ea3-4088-9cdf-e80b0bf2e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:34.240047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7da9d7e6-2ea3-4088-9cdf-e80b0bf2e4a8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:34.241301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7da9d7e6-2ea3-4088-9cdf-e80b0bf2e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:34.263049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:34.343512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:34.385099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889090c-2e20-4262-98fe-b03f2a57e5f7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:34.386637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a889090c-2e20-4262-98fe-b03f2a57e5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:34.494756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889090c-2e20-4262-98fe-b03f2a57e5f7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:34.515081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:34.568826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215f6db-bcfb-4143-a03e-16ae9e6fdac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:34.569499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215f6db-bcfb-4143-a03e-16ae9e6fdac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:34.569780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215f6db-bcfb-4143-a03e-16ae9e6fdac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:34.570055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215f6db-bcfb-4143-a03e-16ae9e6fdac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:34.572830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c215f6db-bcfb-4143-a03e-16ae9e6fdac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:34.683535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215f6db-bcfb-4143-a03e-16ae9e6fdac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:34.685488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215f6db-bcfb-4143-a03e-16ae9e6fdac1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:34.704061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 721/1452] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:36.864858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f5a23-7a5d-4561-89ab-3eeb7156245a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:36.865390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f5a23-7a5d-4561-89ab-3eeb7156245a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:36.865390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f5a23-7a5d-4561-89ab-3eeb7156245a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:36.866707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:36.872725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b49e8-5644-4cf2-a748-d26b90449007 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:36.873153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b49e8-5644-4cf2-a748-d26b90449007 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:36.873283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b49e8-5644-4cf2-a748-d26b90449007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:36.937071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:36.939657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5d0b49e8-5644-4cf2-a748-d26b90449007 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:36.940890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d0b49e8-5644-4cf2-a748-d26b90449007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:37.038598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:37.077818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f5a23-7a5d-4561-89ab-3eeb7156245a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:37.079351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-146f5a23-7a5d-4561-89ab-3eeb7156245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:37.185307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f5a23-7a5d-4561-89ab-3eeb7156245a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:37.235447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:37.260788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97924d12-822f-4ead-bd75-7d867730dd84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:37.261414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924d12-822f-4ead-bd75-7d867730dd84 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:37.261700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924d12-822f-4ead-bd75-7d867730dd84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:37.261977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924d12-822f-4ead-bd75-7d867730dd84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:37.263608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-97924d12-822f-4ead-bd75-7d867730dd84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:37.377969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97924d12-822f-4ead-bd75-7d867730dd84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:37.379407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97924d12-822f-4ead-bd75-7d867730dd84 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:37.398159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 733/1455] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:39.468389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ace8ee-7a06-418a-a1c2-8625d2d1a94a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:39.468846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ace8ee-7a06-418a-a1c2-8625d2d1a94a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:39.468983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ace8ee-7a06-418a-a1c2-8625d2d1a94a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:39.470339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:39.475694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec865955-6f0f-4961-9167-d8737525e8ad None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:39.476101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec865955-6f0f-4961-9167-d8737525e8ad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:39.476245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec865955-6f0f-4961-9167-d8737525e8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:39.542433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ec865955-6f0f-4961-9167-d8737525e8ad None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:39.543657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ec865955-6f0f-4961-9167-d8737525e8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:39.573305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:39.642121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:39.682178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ace8ee-7a06-418a-a1c2-8625d2d1a94a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:39.683708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b6ace8ee-7a06-418a-a1c2-8625d2d1a94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:39.788817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ace8ee-7a06-418a-a1c2-8625d2d1a94a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:39.817121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:39.863581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c45c2-c9fe-4f1c-bcae-6025649f83e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:39.864205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c45c2-c9fe-4f1c-bcae-6025649f83e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:39.864532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c45c2-c9fe-4f1c-bcae-6025649f83e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:39.864815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c45c2-c9fe-4f1c-bcae-6025649f83e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:39.866445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-126c45c2-c9fe-4f1c-bcae-6025649f83e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:39.975558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c45c2-c9fe-4f1c-bcae-6025649f83e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:39.977619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c45c2-c9fe-4f1c-bcae-6025649f83e5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:39.996579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 724/1458] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:42.068262 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f1602-24bd-46db-acff-64e6ab599831 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:42.068845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f1602-24bd-46db-acff-64e6ab599831 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:42.068845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f1602-24bd-46db-acff-64e6ab599831 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:42.070214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:42.076222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83958ead-a51b-4c12-b90d-5709dac0babd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:42.076697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83958ead-a51b-4c12-b90d-5709dac0babd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:42.076838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83958ead-a51b-4c12-b90d-5709dac0babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:42.144144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-83958ead-a51b-4c12-b90d-5709dac0babd None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:42.145382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83958ead-a51b-4c12-b90d-5709dac0babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:42.153921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:42.244389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:42.281545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f1602-24bd-46db-acff-64e6ab599831 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:42.283175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4f8f1602-24bd-46db-acff-64e6ab599831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:42.390506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f1602-24bd-46db-acff-64e6ab599831 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:42.449315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:42.469864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50751a1-52c6-4cb3-8ecd-b15871646215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:42.470486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50751a1-52c6-4cb3-8ecd-b15871646215 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:42.470767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50751a1-52c6-4cb3-8ecd-b15871646215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:42.471042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50751a1-52c6-4cb3-8ecd-b15871646215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:42.472820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d50751a1-52c6-4cb3-8ecd-b15871646215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:42.594669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50751a1-52c6-4cb3-8ecd-b15871646215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:42.596118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50751a1-52c6-4cb3-8ecd-b15871646215 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:42.615413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 736/1461] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:44.697350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2c7c-f689-4e6a-9f18-eb86d3ac8919 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:44.697888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2c7c-f689-4e6a-9f18-eb86d3ac8919 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:44.697981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2c7c-f689-4e6a-9f18-eb86d3ac8919 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:44.699264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:44.704774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248fb49-5101-41c2-b161-4deb9f603d0f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:44.705230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248fb49-5101-41c2-b161-4deb9f603d0f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:44.705341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248fb49-5101-41c2-b161-4deb9f603d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:44.773530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8248fb49-5101-41c2-b161-4deb9f603d0f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:44.774754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8248fb49-5101-41c2-b161-4deb9f603d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:44.813816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:44.871907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:44.909849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a2c7c-f689-4e6a-9f18-eb86d3ac8919 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:44.911413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ca9a2c7c-f689-4e6a-9f18-eb86d3ac8919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:45.017986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a2c7c-f689-4e6a-9f18-eb86d3ac8919 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:45.054702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:45.092231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f8f69-b901-49de-8c7d-9382f411e6fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:45.092887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f8f69-b901-49de-8c7d-9382f411e6fc service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:45.093173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f8f69-b901-49de-8c7d-9382f411e6fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:45.093454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f8f69-b901-49de-8c7d-9382f411e6fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:45.095100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c36f8f69-b901-49de-8c7d-9382f411e6fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:45.205636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f8f69-b901-49de-8c7d-9382f411e6fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:45.207075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f8f69-b901-49de-8c7d-9382f411e6fc service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:45.225778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 727/1464] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:47.306868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5228ebf-6869-4dcb-9dcb-4266f581f726 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:47.307436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5228ebf-6869-4dcb-9dcb-4266f581f726 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:47.307436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5228ebf-6869-4dcb-9dcb-4266f581f726 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:47.308818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:47.314028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353ae47-8032-48ac-906b-416aec1c5c83 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:47.314414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353ae47-8032-48ac-906b-416aec1c5c83 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:47.314595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353ae47-8032-48ac-906b-416aec1c5c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:47.382125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0353ae47-8032-48ac-906b-416aec1c5c83 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:47.383998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0353ae47-8032-48ac-906b-416aec1c5c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:47.403398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:47.486743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:47.523893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5228ebf-6869-4dcb-9dcb-4266f581f726 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:47.525605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f5228ebf-6869-4dcb-9dcb-4266f581f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:47.635745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5228ebf-6869-4dcb-9dcb-4266f581f726 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:47.705388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:47.712105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c841ab-1a72-4663-99bf-17d547775cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:47.717539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c841ab-1a72-4663-99bf-17d547775cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:47.717539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c841ab-1a72-4663-99bf-17d547775cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:47.717539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c841ab-1a72-4663-99bf-17d547775cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:47.717539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-40c841ab-1a72-4663-99bf-17d547775cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:47.825113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c841ab-1a72-4663-99bf-17d547775cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:47.826561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c841ab-1a72-4663-99bf-17d547775cdd service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:47.844896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 739/1467] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:28:49.895181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea098c-88c9-4936-a1d4-60e58470102f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:49.895766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea098c-88c9-4936-a1d4-60e58470102f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:49.895766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea098c-88c9-4936-a1d4-60e58470102f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:49.897125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:28:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:28:49.902491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950700e-3610-4395-9f56-1fae208f92a2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:49.902918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950700e-3610-4395-9f56-1fae208f92a2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:49.903036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950700e-3610-4395-9f56-1fae208f92a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:49.968144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e950700e-3610-4395-9f56-1fae208f92a2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:28:49.969391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e950700e-3610-4395-9f56-1fae208f92a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:50.018635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:50.069282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:28:50.107337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea098c-88c9-4936-a1d4-60e58470102f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:28:50.108912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-19ea098c-88c9-4936-a1d4-60e58470102f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:50.213404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea098c-88c9-4936-a1d4-60e58470102f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:28:50.261388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:28:50.290625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb72e7-ee47-421a-842d-d74c6d7e8bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:28:50.291249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb72e7-ee47-421a-842d-d74c6d7e8bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:28:50.291534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb72e7-ee47-421a-842d-d74c6d7e8bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:28:50.291806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb72e7-ee47-421a-842d-d74c6d7e8bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:28:50.293486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f7bb72e7-ee47-421a-842d-d74c6d7e8bb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:28:50.406762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb72e7-ee47-421a-842d-d74c6d7e8bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:28:50.408211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb72e7-ee47-421a-842d-d74c6d7e8bb3 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:28:50.427735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 730/1470] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:29:00.067458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=22 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:00.425486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=66 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:11.358296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3667e-ab4a-4e2b-abce-82bd459ced52 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:11.358824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3667e-ab4a-4e2b-abce-82bd459ced52 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:11.358824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3667e-ab4a-4e2b-abce-82bd459ced52 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:11.360263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 741/1471] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:29:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:29:11.365507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a6a12-1e4b-4453-917b-e20b8fbf8baf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:11.366027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a6a12-1e4b-4453-917b-e20b8fbf8baf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:11.366740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a6a12-1e4b-4453-917b-e20b8fbf8baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:11.431935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-788a6a12-1e4b-4453-917b-e20b8fbf8baf None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:11.433179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-788a6a12-1e4b-4453-917b-e20b8fbf8baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:11.531041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 731/1472] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:29:11.616995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512bd3-7f27-456f-b588-cdaa77e8eb51 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:11.617287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512bd3-7f27-456f-b588-cdaa77e8eb51 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:11.617569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512bd3-7f27-456f-b588-cdaa77e8eb51 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:11.618907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 742/1473] 10.222.0.128 () {54 vars in 971 bytes} [Tue Jul 26 15:29:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:29:11.623661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9250-4fc7-4a00-9b31-ae3bf83c63fb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:11.623958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9250-4fc7-4a00-9b31-ae3bf83c63fb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:11.624241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9250-4fc7-4a00-9b31-ae3bf83c63fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:11.821066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4f7b9250-4fc7-4a00-9b31-ae3bf83c63fb None None] MFA Rules not processed for user `0207287721f5489f8ac7a258356e57bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:11.822294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f7b9250-4fc7-4a00-9b31-ae3bf83c63fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:11.843079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:11.900872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 732/1474] 10.222.0.128 () {58 vars in 1068 bytes} [Tue Jul 26 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:29:11.908495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512bd3-7f27-456f-b588-cdaa77e8eb51 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:29:11.910013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-66512bd3-7f27-456f-b588-cdaa77e8eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:12.012333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512bd3-7f27-456f-b588-cdaa77e8eb51 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:29:12.084570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fff7b7-8530-42c0-9b81-4a6976df92f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:29:12.085273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff7b7-8530-42c0-9b81-4a6976df92f8 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:12.085556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff7b7-8530-42c0-9b81-4a6976df92f8 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:12.085833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff7b7-8530-42c0-9b81-4a6976df92f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:12.087476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-44fff7b7-8530-42c0-9b81-4a6976df92f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:12.140801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:12.197099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fff7b7-8530-42c0-9b81-4a6976df92f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:29:12.198468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fff7b7-8530-42c0-9b81-4a6976df92f8 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:29:12.216694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 743/1475] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:29:14.352588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438b8a-50a9-4428-990b-cef368de7141 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:14.353202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438b8a-50a9-4428-990b-cef368de7141 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:14.353335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438b8a-50a9-4428-990b-cef368de7141 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:14.354888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 733/1476] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:29:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:29:14.360420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b537714-59c9-4614-8550-18805cc4096e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:14.360936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b537714-59c9-4614-8550-18805cc4096e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:14.361031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b537714-59c9-4614-8550-18805cc4096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:14.431547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9b537714-59c9-4614-8550-18805cc4096e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:14.432857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9b537714-59c9-4614-8550-18805cc4096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:14.532057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 744/1477] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:29:14.555276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e438b8a-50a9-4428-990b-cef368de7141 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:29:14.556884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e438b8a-50a9-4428-990b-cef368de7141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:14.689026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e438b8a-50a9-4428-990b-cef368de7141 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:29:14.739384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:14.785005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe055141-10c4-453f-a9a6-0f0a17204241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:29:14.785616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe055141-10c4-453f-a9a6-0f0a17204241 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:14.785899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe055141-10c4-453f-a9a6-0f0a17204241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:14.786174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe055141-10c4-453f-a9a6-0f0a17204241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:14.786870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe055141-10c4-453f-a9a6-0f0a17204241 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:29:14.788665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-fe055141-10c4-453f-a9a6-0f0a17204241 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:29:14.788665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:29:14.788665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:29:14.790037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe055141-10c4-453f-a9a6-0f0a17204241 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:29:14.809142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 734/1478] 10.222.0.128 () {56 vars in 1113 bytes} [Tue Jul 26 15:29:14 2022] GET /identity/v3/projects => generated 2562 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:29:16.879776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6569db-0335-43ec-a07a-d5a7de51d0d4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:16.880334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6569db-0335-43ec-a07a-d5a7de51d0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:16.880334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6569db-0335-43ec-a07a-d5a7de51d0d4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:16.882482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 745/1479] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:29:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:29:16.887842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b109b2-58ff-4619-9d02-9d66f3d77da2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:16.888266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b109b2-58ff-4619-9d02-9d66f3d77da2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:16.888402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b109b2-58ff-4619-9d02-9d66f3d77da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:16.961955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-04b109b2-58ff-4619-9d02-9d66f3d77da2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:16.963236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-04b109b2-58ff-4619-9d02-9d66f3d77da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:17.065650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 735/1480] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:29:17.110017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6569db-0335-43ec-a07a-d5a7de51d0d4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:29:17.111588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5a6569db-0335-43ec-a07a-d5a7de51d0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:17.122984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:17.264418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6569db-0335-43ec-a07a-d5a7de51d0d4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:29:17.345942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:17.379519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df17e9-6143-4e6b-90e8-b42eceba70c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:29:17.380143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df17e9-6143-4e6b-90e8-b42eceba70c5 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:17.380433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df17e9-6143-4e6b-90e8-b42eceba70c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:17.380742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df17e9-6143-4e6b-90e8-b42eceba70c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:17.382972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-92df17e9-6143-4e6b-90e8-b42eceba70c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:17.473782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df17e9-6143-4e6b-90e8-b42eceba70c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:29:17.476008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df17e9-6143-4e6b-90e8-b42eceba70c5 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:29:17.494513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 746/1481] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:29:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:29:19.762471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd859051-9ad3-407c-9a70-24fad5d52ce2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:19.763086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd859051-9ad3-407c-9a70-24fad5d52ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:19.763086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd859051-9ad3-407c-9a70-24fad5d52ce2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:19.764478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 736/1482] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:29:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:29:19.769874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b83638-cb03-4a45-92c8-be04f8479000 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:19.770320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b83638-cb03-4a45-92c8-be04f8479000 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:19.770461 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b83638-cb03-4a45-92c8-be04f8479000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:19.839420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-09b83638-cb03-4a45-92c8-be04f8479000 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:19.840671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-09b83638-cb03-4a45-92c8-be04f8479000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:19.888376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:19.947729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 747/1483] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:29:19.984031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd859051-9ad3-407c-9a70-24fad5d52ce2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:29:19.985590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fd859051-9ad3-407c-9a70-24fad5d52ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:20.009772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:20.091408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd859051-9ad3-407c-9a70-24fad5d52ce2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:29:20.167576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2d367-4bbd-44d1-bbd7-6ccb624f5950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:29:20.168195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2d367-4bbd-44d1-bbd7-6ccb624f5950 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:20.168503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2d367-4bbd-44d1-bbd7-6ccb624f5950 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:20.168788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2d367-4bbd-44d1-bbd7-6ccb624f5950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:20.170436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-08a2d367-4bbd-44d1-bbd7-6ccb624f5950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:20.244777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:20.280550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2d367-4bbd-44d1-bbd7-6ccb624f5950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:29:20.281998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2d367-4bbd-44d1-bbd7-6ccb624f5950 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:29:20.300058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 737/1484] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:29:22.439499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b083-6298-4f0a-a322-ad80eb7f7eca None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:22.440089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b083-6298-4f0a-a322-ad80eb7f7eca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:22.440089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b083-6298-4f0a-a322-ad80eb7f7eca None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:22.441483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 748/1485] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:29:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:29:22.447025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062720a-c5e6-4a91-97e4-8b718c3e4ac8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:22.447450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062720a-c5e6-4a91-97e4-8b718c3e4ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:22.447543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062720a-c5e6-4a91-97e4-8b718c3e4ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:22.515683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6062720a-c5e6-4a91-97e4-8b718c3e4ac8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:22.516858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6062720a-c5e6-4a91-97e4-8b718c3e4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:22.625342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 738/1486] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:29:22.665552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b083-6298-4f0a-a322-ad80eb7f7eca None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:29:22.667092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4d35b083-6298-4f0a-a322-ad80eb7f7eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:22.777000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b083-6298-4f0a-a322-ad80eb7f7eca None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:29:22.850726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:22.852754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d410b18-e176-43f2-a86f-c784e9ef7658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:29:22.853420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d410b18-e176-43f2-a86f-c784e9ef7658 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:22.853742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d410b18-e176-43f2-a86f-c784e9ef7658 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:22.854177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d410b18-e176-43f2-a86f-c784e9ef7658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:22.855759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2d410b18-e176-43f2-a86f-c784e9ef7658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:22.967346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d410b18-e176-43f2-a86f-c784e9ef7658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:29:22.968844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d410b18-e176-43f2-a86f-c784e9ef7658 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:29:22.987501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 749/1487] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:29:25.211077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109ed2-d608-440c-bf99-dbb4dc5f1696 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:25.211653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109ed2-d608-440c-bf99-dbb4dc5f1696 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:25.211653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109ed2-d608-440c-bf99-dbb4dc5f1696 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:25.213076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 739/1488] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:29:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:29:25.218457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56c4cb-ca8f-46e7-b421-96cbce73c8e7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:25.218886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56c4cb-ca8f-46e7-b421-96cbce73c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:25.219019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56c4cb-ca8f-46e7-b421-96cbce73c8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:25.292708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cd56c4cb-ca8f-46e7-b421-96cbce73c8e7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:25.294096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cd56c4cb-ca8f-46e7-b421-96cbce73c8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:25.370163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:25.419071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 750/1489] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:29:25.462227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6109ed2-d608-440c-bf99-dbb4dc5f1696 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:29:25.463828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c6109ed2-d608-440c-bf99-dbb4dc5f1696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:25.502058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:25.572920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6109ed2-d608-440c-bf99-dbb4dc5f1696 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:29:25.646259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6902c9d-73a4-4e6c-ab14-6de483ab41c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:29:25.646984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902c9d-73a4-4e6c-ab14-6de483ab41c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:25.647850 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902c9d-73a4-4e6c-ab14-6de483ab41c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:25.648143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902c9d-73a4-4e6c-ab14-6de483ab41c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:25.649874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e6902c9d-73a4-4e6c-ab14-6de483ab41c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:25.744187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:25.776237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902c9d-73a4-4e6c-ab14-6de483ab41c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:29:25.777853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902c9d-73a4-4e6c-ab14-6de483ab41c5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:29:25.799475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 740/1490] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:29:28.243502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a5c3a-2c9f-4503-8d0f-2fb1f46e3e66 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:28.244081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a5c3a-2c9f-4503-8d0f-2fb1f46e3e66 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:28.244081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a5c3a-2c9f-4503-8d0f-2fb1f46e3e66 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:28.245463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 751/1491] 10.222.0.128 () {54 vars in 931 bytes} [Tue Jul 26 15:29:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:29:28.250700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864217-5761-45a9-a001-29b7c2eb515a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:28.251119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864217-5761-45a9-a001-29b7c2eb515a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:28.251295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864217-5761-45a9-a001-29b7c2eb515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:28.317117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-38864217-5761-45a9-a001-29b7c2eb515a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:29:28.318327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-38864217-5761-45a9-a001-29b7c2eb515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:28.417817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 741/1492] 10.222.0.128 () {58 vars in 1028 bytes} [Tue Jul 26 15:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:29:28.457800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a5c3a-2c9f-4503-8d0f-2fb1f46e3e66 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:29:28.459383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e5a5c3a-2c9f-4503-8d0f-2fb1f46e3e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:28.564254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a5c3a-2c9f-4503-8d0f-2fb1f46e3e66 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:29:28.636636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfeb28-9d58-4cbf-947b-85d6573c0063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:29:28.637252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfeb28-9d58-4cbf-947b-85d6573c0063 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:29:28.637557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfeb28-9d58-4cbf-947b-85d6573c0063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:29:28.637829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfeb28-9d58-4cbf-947b-85d6573c0063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:29:28.639471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2cbfeb28-9d58-4cbf-947b-85d6573c0063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:29:28.649975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:28.751173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfeb28-9d58-4cbf-947b-85d6573c0063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:29:28.752655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfeb28-9d58-4cbf-947b-85d6573c0063 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:29:28.771265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 752/1493] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:29:38.416063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=82 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:29:38.768678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=50 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:13.498629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:13.499221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:13.499221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:13.577451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:31:13.578696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:13.677925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 742/1494] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:31:13.686215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71ce3a-2ab3-4be3-8e62-6d665567e65b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:13.686698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71ce3a-2ab3-4be3-8e62-6d665567e65b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:13.686802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71ce3a-2ab3-4be3-8e62-6d665567e65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:13.762387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ea71ce3a-2ab3-4be3-8e62-6d665567e65b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:31:13.763510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ea71ce3a-2ab3-4be3-8e62-6d665567e65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:13.863014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 753/1495] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:31:13.868028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:13.869277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:13.881200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36080de-f245-46d3-9041-7bccc5d174ef None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:13.942080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:13.984296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:13.985007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:13.985323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:13.985601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:13.986310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:31:13.986611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:13.987616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:13.987616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:31:13.987616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:31:13.988518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:14.000297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 743/1496] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:31:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:31:14.009572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71ce3a-2ab3-4be3-8e62-6d665567e65b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:14.018699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71ce3a-2ab3-4be3-8e62-6d665567e65b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:14.081767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:14.118025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:14.118639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:14.118919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:14.119192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:14.119857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:14.121652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:14.121652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Jul 26 15:31:14.121652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: Jul 26 15:31:14.122845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:14.305787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 754/1497] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:31:14 2022] POST /identity/v3/projects => generated 378 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:31:14.309931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:14.311514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:14.346687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1dde6-b4dc-4faf-86e9-9635bfc45f09 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:14.444128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:14.444781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:14.445065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:14.445342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:14.446063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:14.447074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:14.447074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:31:14.447074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:31:14.448283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:14.605397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 744/1498] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:31:14 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:31:14.609458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:14.611001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:14.676211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46396dbb-e991-45dd-b9db-38b58d617b14 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:14.763338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:14.774614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:14.778032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:14.778032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:14.778032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:14.778032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:14.778032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:14.778032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Jul 26 15:31:14.778032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: Jul 26 15:31:14.778631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:14.791802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 755/1499] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:31:14 2022] GET /identity/v3/roles => generated 1427 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:31:14.796162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:14.797750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:14.809435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12d394-041c-46cf-b291-0e81c0badf40 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:14.838351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:14.911634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0185cb-4dce-42bd-b55e-abad27aa1a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:14.912826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0185cb-4dce-42bd-b55e-abad27aa1a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:14.913105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0185cb-4dce-42bd-b55e-abad27aa1a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:14.913379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0185cb-4dce-42bd-b55e-abad27aa1a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:14.914128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0185cb-4dce-42bd-b55e-abad27aa1a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:14.915226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-bf0185cb-4dce-42bd-b55e-abad27aa1a04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:14.915226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:31:14.915226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:31:14.916149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0185cb-4dce-42bd-b55e-abad27aa1a04 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:14.929509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 745/1500] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:31:14 2022] GET /identity/v3/roles => generated 1427 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:31:14.933778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:14.935315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:14.969753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975b26-bede-48d3-9f2d-51e3a9e9c3ab admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:15.064760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9a28c-4566-4cdd-ba75-11047c7abfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:15.065550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9a28c-4566-4cdd-ba75-11047c7abfbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:15.065711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9a28c-4566-4cdd-ba75-11047c7abfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:15.065993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9a28c-4566-4cdd-ba75-11047c7abfbc admin admin] PATH_INFO: `/v3/projects/7f20aed89e2a4b478f9a06e40481866f/users/a74f80f987ff4445b73b9b206fe40353/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:15.089087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9a28c-4566-4cdd-ba75-11047c7abfbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f20aed89e2a4b478f9a06e40481866f, user_id=a74f80f987ff4445b73b9b206fe40353, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:15.090175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-d7c9a28c-4566-4cdd-ba75-11047c7abfbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:15.090175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Jul 26 15:31:15.090175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: Jul 26 15:31:15.092408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9a28c-4566-4cdd-ba75-11047c7abfbc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:15.199215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 756/1501] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:31:14 2022] PUT /identity/v3/projects/7f20aed89e2a4b478f9a06e40481866f/users/a74f80f987ff4445b73b9b206fe40353/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:31:15.212024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c14ab-7416-4eee-a371-014eded046f8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:15.212313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c14ab-7416-4eee-a371-014eded046f8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:15.212629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c14ab-7416-4eee-a371-014eded046f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:15.273226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-887c14ab-7416-4eee-a371-014eded046f8 None None] MFA Rules not processed for user `a74f80f987ff4445b73b9b206fe40353`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:31:15.274515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-887c14ab-7416-4eee-a371-014eded046f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:15.354302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 746/1502] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:31:15.368059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528d013-f91a-479a-86a2-09eff3ad25e9 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:15.368352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528d013-f91a-479a-86a2-09eff3ad25e9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:15.368676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528d013-f91a-479a-86a2-09eff3ad25e9 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:15.370109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 757/1503] 10.222.0.128 () {54 vars in 969 bytes} [Tue Jul 26 15:31:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:31:15.375059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6f8a6-e168-419a-8eac-06e4f683d805 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:15.375352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6f8a6-e168-419a-8eac-06e4f683d805 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:15.375630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6f8a6-e168-419a-8eac-06e4f683d805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:15.408165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:15.452172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-56e6f8a6-e168-419a-8eac-06e4f683d805 None None] MFA Rules not processed for user `8ff6f527aa364534b2fdd0b770924e83`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:31:15.454161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-56e6f8a6-e168-419a-8eac-06e4f683d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:15.540805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 747/1504] 10.222.0.128 () {58 vars in 1066 bytes} [Tue Jul 26 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:31:15.548843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528d013-f91a-479a-86a2-09eff3ad25e9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:15.550375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0528d013-f91a-479a-86a2-09eff3ad25e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:15.601872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:15.656038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528d013-f91a-479a-86a2-09eff3ad25e9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:15.730603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:15.731231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:15.731516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:15.731784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:15.733464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:15.836583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:15.838105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:15.842271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:15.857751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 758/1505] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:31:15.904764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a147a7-be62-4bc0-be87-80bb33244b95 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:15.905238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a147a7-be62-4bc0-be87-80bb33244b95 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:15.905377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a147a7-be62-4bc0-be87-80bb33244b95 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:15.906594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 748/1506] 10.222.0.128 () {52 vars in 835 bytes} [Tue Jul 26 15:31:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:31:15.910250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:15.911570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:16.013853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e025-ea2b-4c49-911b-8ec617ee2962 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:16.112628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:16.113276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:16.113564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:16.113847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] PATH_INFO: `/v3/users/a74f80f987ff4445b73b9b206fe40353` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:16.128745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a74f80f987ff4445b73b9b206fe40353)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:16.129791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:16.129791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Jul 26 15:31:16.129791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: Jul 26 15:31:16.131018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:16.141840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:16.404658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a74f80f987ff4445b73b9b206fe40353'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:16.405181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a74f80f987ff4445b73b9b206fe40353'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:16.418554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a74f80f987ff4445b73b9b206fe40353'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:16.438017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a74f80f987ff4445b73b9b206fe40353'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:16.499422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a74f80f987ff4445b73b9b206fe40353'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:16.505729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 759/1507] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:31:15 2022] DELETE /identity/v3/users/a74f80f987ff4445b73b9b206fe40353 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:31:16.510657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:16.510966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:16.511247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:16.569556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:31:16.570663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:16.651702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:16.664999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 749/1508] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:31:16.674780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:16.676051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:16.780536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fcc95-86d0-43f2-a0d1-4e0df18b18a1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:16.838165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:16.858004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092655e-0432-426f-b491-a25f8bf71f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:16.858685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092655e-0432-426f-b491-a25f8bf71f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:16.858971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092655e-0432-426f-b491-a25f8bf71f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:16.859249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092655e-0432-426f-b491-a25f8bf71f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:16.860962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a092655e-0432-426f-b491-a25f8bf71f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:16.973528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092655e-0432-426f-b491-a25f8bf71f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:16.974927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092655e-0432-426f-b491-a25f8bf71f64 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:16.994889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 760/1509] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:31:17.468113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:31:17.469480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:31:17.481069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed95446-e604-4de5-a9c2-8b64a2b27eab None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:31:17.578924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:31:17.579570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:31:17.579854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:31:17.580129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] PATH_INFO: `/v3/projects/7f20aed89e2a4b478f9a06e40481866f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:31:17.590677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f20aed89e2a4b478f9a06e40481866f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:31:17.591978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:31:17.591978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:31:17.591978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:31:17.593215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:31:17.716839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f20aed89e2a4b478f9a06e40481866f', 'request_id': 'req-b235d882-10b9-4c7e-a646-d34d357af7df'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:17.802619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f20aed89e2a4b478f9a06e40481866f', 'request_id': 'req-b235d882-10b9-4c7e-a646-d34d357af7df'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:17.809640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f20aed89e2a4b478f9a06e40481866f', 'request_id': 'req-b235d882-10b9-4c7e-a646-d34d357af7df'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:17.841743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] The token cache is being invalidate because project 7f20aed89e2a4b478f9a06e40481866f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:31:17.842033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:17.842566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b235d882-10b9-4c7e-a646-d34d357af7df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:31:17.843934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 750/1510] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:31:17 2022] DELETE /identity/v3/projects/7f20aed89e2a4b478f9a06e40481866f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:31:26.992614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=62 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:31:27.839309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:33:01.883481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051988-9466-422c-ba30-722e2e66d283 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:01.884054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051988-9466-422c-ba30-722e2e66d283 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:01.884054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051988-9466-422c-ba30-722e2e66d283 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:01.885514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:33:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:01.890900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ea384-9e0a-4fc3-9a36-aa68b5e63a7e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:01.891386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ea384-9e0a-4fc3-9a36-aa68b5e63a7e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:01.891494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ea384-9e0a-4fc3-9a36-aa68b5e63a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:01.976410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9e1ea384-9e0a-4fc3-9a36-aa68b5e63a7e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:01.977684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9e1ea384-9e0a-4fc3-9a36-aa68b5e63a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:02.087731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:33:02.245293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e051988-9466-422c-ba30-722e2e66d283 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:02.247552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7e051988-9466-422c-ba30-722e2e66d283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:02.372333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e051988-9466-422c-ba30-722e2e66d283 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:02.377934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a6eff-cb83-465b-9b8f-f18258197c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:02.378549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6eff-cb83-465b-9b8f-f18258197c0a service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:02.378830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6eff-cb83-465b-9b8f-f18258197c0a service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:02.379107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6eff-cb83-465b-9b8f-f18258197c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:02.388011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a6eff-cb83-465b-9b8f-f18258197c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:02.389649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a6eff-cb83-465b-9b8f-f18258197c0a service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:02.399345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 762/1513] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:03.344681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34837c-ffc4-482c-934d-508a4f386769 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:03.345227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34837c-ffc4-482c-934d-508a4f386769 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:03.345227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34837c-ffc4-482c-934d-508a4f386769 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:03.346567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:33:03 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:03.353489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f46a2-059f-4407-b314-40f9cfd2275a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:03.354287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f46a2-059f-4407-b314-40f9cfd2275a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:03.354287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f46a2-059f-4407-b314-40f9cfd2275a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:03.376109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-868f46a2-059f-4407-b314-40f9cfd2275a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:03.377367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-868f46a2-059f-4407-b314-40f9cfd2275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:03.385697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:03.545226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34837c-ffc4-482c-934d-508a4f386769 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:03.560897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34837c-ffc4-482c-934d-508a4f386769 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:03.567156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0c37a-7a1c-49cf-bcc5-038ecb5d3f08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:03.567772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0c37a-7a1c-49cf-bcc5-038ecb5d3f08 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:03.568060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0c37a-7a1c-49cf-bcc5-038ecb5d3f08 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:03.568337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0c37a-7a1c-49cf-bcc5-038ecb5d3f08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:03.576924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0c37a-7a1c-49cf-bcc5-038ecb5d3f08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:03.578525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0c37a-7a1c-49cf-bcc5-038ecb5d3f08 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:03.588664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 753/1516] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:04.311630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad192c1-e7ce-4233-b62f-51e3db604837 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:04.311913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad192c1-e7ce-4233-b62f-51e3db604837 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:04.312191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad192c1-e7ce-4233-b62f-51e3db604837 None None] PATH_INFO: `/v3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:04.313582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 764/1517] 10.222.0.48 () {54 vars in 929 bytes} [Tue Jul 26 15:33:04 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:33:04.320832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a092d7-687b-423b-8c11-2654c93654d7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:04.321126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a092d7-687b-423b-8c11-2654c93654d7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:04.321412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a092d7-687b-423b-8c11-2654c93654d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:04.383125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-68a092d7-687b-423b-8c11-2654c93654d7 None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:04.384348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-68a092d7-687b-423b-8c11-2654c93654d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:04.401892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:33:04.456898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 754/1518] 10.222.0.48 () {58 vars in 1019 bytes} [Tue Jul 26 15:33:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:08.683404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aefdd0-359e-45ba-96e8-38763921560f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:08.683941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aefdd0-359e-45ba-96e8-38763921560f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:08.683941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aefdd0-359e-45ba-96e8-38763921560f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:08.685376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 765/1519] 10.222.0.128 () {54 vars in 966 bytes} [Tue Jul 26 15:33:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:08.690288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cc08e-150d-45cb-a16d-fef16e67284b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:08.690711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cc08e-150d-45cb-a16d-fef16e67284b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:08.690858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cc08e-150d-45cb-a16d-fef16e67284b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:08.710998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-135cc08e-150d-45cb-a16d-fef16e67284b None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:08.712241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-135cc08e-150d-45cb-a16d-fef16e67284b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:08.720884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 755/1520] 10.222.0.128 () {58 vars in 1063 bytes} [Tue Jul 26 15:33:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:08.729665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aefdd0-359e-45ba-96e8-38763921560f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:08.743981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aefdd0-359e-45ba-96e8-38763921560f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:08.749333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c49f5-3540-4fac-9eec-ff514bb1c681 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:08.749947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c49f5-3540-4fac-9eec-ff514bb1c681 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:08.750231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c49f5-3540-4fac-9eec-ff514bb1c681 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:08.750510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c49f5-3540-4fac-9eec-ff514bb1c681 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:08.759355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c49f5-3540-4fac-9eec-ff514bb1c681 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:08.761007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c49f5-3540-4fac-9eec-ff514bb1c681 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:08.770873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 766/1521] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:14.844570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa1c5-4a85-476b-8cd6-5d6542351954 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:14.845163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa1c5-4a85-476b-8cd6-5d6542351954 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:14.845163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa1c5-4a85-476b-8cd6-5d6542351954 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:14.846409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:33:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:14.851985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483238-958f-448a-8413-f0ca09236fdc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:14.852435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483238-958f-448a-8413-f0ca09236fdc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:14.852655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483238-958f-448a-8413-f0ca09236fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:14.872484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-91483238-958f-448a-8413-f0ca09236fdc None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:14.873732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-91483238-958f-448a-8413-f0ca09236fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:14.881599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:15.038802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433aa1c5-4a85-476b-8cd6-5d6542351954 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:15.053289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433aa1c5-4a85-476b-8cd6-5d6542351954 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:15.059623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d91878-9a1e-42d5-b465-c85582ae2da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:15.060271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d91878-9a1e-42d5-b465-c85582ae2da9 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:15.060581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d91878-9a1e-42d5-b465-c85582ae2da9 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:15.060863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d91878-9a1e-42d5-b465-c85582ae2da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:15.070366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d91878-9a1e-42d5-b465-c85582ae2da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:15.071970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d91878-9a1e-42d5-b465-c85582ae2da9 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:15.081823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 757/1524] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:15.929126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b195ce0-0433-48b5-b4df-3173bb95a7f4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:15.929989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b195ce0-0433-48b5-b4df-3173bb95a7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:15.929989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b195ce0-0433-48b5-b4df-3173bb95a7f4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:15.931132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:33:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:15.936089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31fb7-aa37-47fd-9d59-4f54700c4fcc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:15.936553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31fb7-aa37-47fd-9d59-4f54700c4fcc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:15.936687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31fb7-aa37-47fd-9d59-4f54700c4fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:15.956138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a1b31fb7-aa37-47fd-9d59-4f54700c4fcc None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:15.957378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a1b31fb7-aa37-47fd-9d59-4f54700c4fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:15.966132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:16.121218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b195ce0-0433-48b5-b4df-3173bb95a7f4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:16.136265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b195ce0-0433-48b5-b4df-3173bb95a7f4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:16.141864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286372eb-475e-4ac9-812d-d9116ec71ba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:16.142493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286372eb-475e-4ac9-812d-d9116ec71ba7 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:16.142771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286372eb-475e-4ac9-812d-d9116ec71ba7 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:16.143049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286372eb-475e-4ac9-812d-d9116ec71ba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:16.152654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286372eb-475e-4ac9-812d-d9116ec71ba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:16.154280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286372eb-475e-4ac9-812d-d9116ec71ba7 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:16.165666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 769/1527] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:20.300805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f879e5-59a2-4591-83a5-91415c509fd6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:20.301385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f879e5-59a2-4591-83a5-91415c509fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:20.301385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f879e5-59a2-4591-83a5-91415c509fd6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:20.302714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:33:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:20.308705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddb678-79b6-44af-90c0-42184d5e4b5a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:20.309173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddb678-79b6-44af-90c0-42184d5e4b5a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:20.309271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddb678-79b6-44af-90c0-42184d5e4b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:20.329862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5cddb678-79b6-44af-90c0-42184d5e4b5a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:20.331059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5cddb678-79b6-44af-90c0-42184d5e4b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:20.340946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:20.416121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f879e5-59a2-4591-83a5-91415c509fd6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:20.417916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a4f879e5-59a2-4591-83a5-91415c509fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:20.497272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f879e5-59a2-4591-83a5-91415c509fd6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:20.500844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841d5cd-cf53-459b-843b-098275463b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:20.501456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841d5cd-cf53-459b-843b-098275463b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:20.501765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841d5cd-cf53-459b-843b-098275463b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:20.502040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841d5cd-cf53-459b-843b-098275463b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:20.511786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841d5cd-cf53-459b-843b-098275463b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:20.513267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841d5cd-cf53-459b-843b-098275463b2f service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:20.523246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 760/1530] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:22.527580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8956789-c0ba-4138-8ab9-dcecd5185eef None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:22.528147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8956789-c0ba-4138-8ab9-dcecd5185eef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:22.528261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8956789-c0ba-4138-8ab9-dcecd5185eef None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:22.529591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:33:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:22.535004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db671d-92e5-4c5e-bb03-1fea3dc2093a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:22.535452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db671d-92e5-4c5e-bb03-1fea3dc2093a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:22.535551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db671d-92e5-4c5e-bb03-1fea3dc2093a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:22.555233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-19db671d-92e5-4c5e-bb03-1fea3dc2093a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:22.556437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-19db671d-92e5-4c5e-bb03-1fea3dc2093a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:22.564198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:22.600201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8956789-c0ba-4138-8ab9-dcecd5185eef None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:22.611032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8956789-c0ba-4138-8ab9-dcecd5185eef None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:22.614536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cc964-ea37-454b-9b5e-852ccc0be730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:22.615141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cc964-ea37-454b-9b5e-852ccc0be730 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:22.615931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cc964-ea37-454b-9b5e-852ccc0be730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:22.616260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cc964-ea37-454b-9b5e-852ccc0be730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:22.625361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cc964-ea37-454b-9b5e-852ccc0be730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:22.626838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cc964-ea37-454b-9b5e-852ccc0be730 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:22.637402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 772/1533] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:26.499895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600229f-3c3d-4f3f-8aa4-7bdd4dc953b4 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:26.500485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600229f-3c3d-4f3f-8aa4-7bdd4dc953b4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:26.500485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600229f-3c3d-4f3f-8aa4-7bdd4dc953b4 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:26.501911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:33:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:26.507521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7573c-7318-46ca-8e96-6af8f36b9dc3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:26.507957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7573c-7318-46ca-8e96-6af8f36b9dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:26.508061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7573c-7318-46ca-8e96-6af8f36b9dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:26.528206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-96d7573c-7318-46ca-8e96-6af8f36b9dc3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:26.529445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96d7573c-7318-46ca-8e96-6af8f36b9dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:26.538325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:26.883786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600229f-3c3d-4f3f-8aa4-7bdd4dc953b4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:26.887378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d600229f-3c3d-4f3f-8aa4-7bdd4dc953b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:26.954358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:33:26.968662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600229f-3c3d-4f3f-8aa4-7bdd4dc953b4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:26.972421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d00d5f-a5cc-4965-a489-21f38e7bdde9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:26.973070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d00d5f-a5cc-4965-a489-21f38e7bdde9 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:26.973352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d00d5f-a5cc-4965-a489-21f38e7bdde9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:26.973631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d00d5f-a5cc-4965-a489-21f38e7bdde9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:26.983848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d00d5f-a5cc-4965-a489-21f38e7bdde9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:26.985359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d00d5f-a5cc-4965-a489-21f38e7bdde9 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:26.995915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 763/1536] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:29.117327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622a4f-b2de-403e-8139-6758761e5bc2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:29.117920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622a4f-b2de-403e-8139-6758761e5bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:29.117920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622a4f-b2de-403e-8139-6758761e5bc2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:29.119385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:33:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:33:29.125174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c6aee-80a6-4870-932f-e5ace4542ba7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:29.125711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c6aee-80a6-4870-932f-e5ace4542ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:29.125816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c6aee-80a6-4870-932f-e5ace4542ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:29.146065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-564c6aee-80a6-4870-932f-e5ace4542ba7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:33:29.147333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-564c6aee-80a6-4870-932f-e5ace4542ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:29.155281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:33:29.473482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b622a4f-b2de-403e-8139-6758761e5bc2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:33:29.476614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8b622a4f-b2de-403e-8139-6758761e5bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:33:29.484271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:33:29.490192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b622a4f-b2de-403e-8139-6758761e5bc2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:33:29.494267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd44afd-48f5-4fb1-a277-5692b08b87cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:33:29.500799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44afd-48f5-4fb1-a277-5692b08b87cd service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:33:29.500799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44afd-48f5-4fb1-a277-5692b08b87cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:33:29.500799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44afd-48f5-4fb1-a277-5692b08b87cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:33:29.505519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd44afd-48f5-4fb1-a277-5692b08b87cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:33:29.507622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd44afd-48f5-4fb1-a277-5692b08b87cd service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:33:29.517921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 775/1539] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:33:39.140435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=11 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:33:39.525799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:02.277993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15829426-49db-4724-8baf-33f805e7d0cf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.279173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15829426-49db-4724-8baf-33f805e7d0cf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.279689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15829426-49db-4724-8baf-33f805e7d0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.311489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-15829426-49db-4724-8baf-33f805e7d0cf None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.328849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15829426-49db-4724-8baf-33f805e7d0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.329778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 765/1540] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.339073 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e042a-099b-4b0a-8681-611491243faa None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.339673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e042a-099b-4b0a-8681-611491243faa None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.339673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e042a-099b-4b0a-8681-611491243faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.373549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d81e042a-099b-4b0a-8681-611491243faa None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.380503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d81e042a-099b-4b0a-8681-611491243faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.399550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 776/1541] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.408245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15829426-49db-4724-8baf-33f805e7d0cf None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:02.422353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15829426-49db-4724-8baf-33f805e7d0cf None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:02.433264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea838a3-b082-4b25-aaf8-72b97c1b1df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:02.434191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea838a3-b082-4b25-aaf8-72b97c1b1df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.434647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea838a3-b082-4b25-aaf8-72b97c1b1df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.435102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea838a3-b082-4b25-aaf8-72b97c1b1df9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.435988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea838a3-b082-4b25-aaf8-72b97c1b1df9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:02.436479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea838a3-b082-4b25-aaf8-72b97c1b1df9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:02.439007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea838a3-b082-4b25-aaf8-72b97c1b1df9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:02.471966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 766/1542] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:02.480113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e042a-099b-4b0a-8681-611491243faa None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:02.494340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e042a-099b-4b0a-8681-611491243faa None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:02.504283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b7d7c-2bc4-4f56-b616-c4c34549e610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:02.505298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b7d7c-2bc4-4f56-b616-c4c34549e610 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.505798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b7d7c-2bc4-4f56-b616-c4c34549e610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.506254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b7d7c-2bc4-4f56-b616-c4c34549e610 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.507811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b7d7c-2bc4-4f56-b616-c4c34549e610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:02.510234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b7d7c-2bc4-4f56-b616-c4c34549e610 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:02.586684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9d75-7ccb-44bd-81b2-1a8d089688da None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.586903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9d75-7ccb-44bd-81b2-1a8d089688da None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.587168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9d75-7ccb-44bd-81b2-1a8d089688da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.617294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 777/1543] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/projects => generated 388 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:02.622350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d3fe9d75-7ccb-44bd-81b2-1a8d089688da None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.622350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d3fe9d75-7ccb-44bd-81b2-1a8d089688da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.640972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e29f3-5049-47db-a83a-2db4492a1f5f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.642302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e29f3-5049-47db-a83a-2db4492a1f5f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.642830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e29f3-5049-47db-a83a-2db4492a1f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.681604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1e5e29f3-5049-47db-a83a-2db4492a1f5f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.683186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1e5e29f3-5049-47db-a83a-2db4492a1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.716275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 767/1544] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.717180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe9d75-7ccb-44bd-81b2-1a8d089688da None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:02.730594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe9d75-7ccb-44bd-81b2-1a8d089688da None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:02.735882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cca94e-b390-441d-8932-0ff96384406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:02.736642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cca94e-b390-441d-8932-0ff96384406e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.736935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cca94e-b390-441d-8932-0ff96384406e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.737210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cca94e-b390-441d-8932-0ff96384406e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.737888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cca94e-b390-441d-8932-0ff96384406e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:02.739936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cca94e-b390-441d-8932-0ff96384406e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:02.781373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 778/1545] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:02.783083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede68723-07eb-4886-bd27-360cdd308357 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.783374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede68723-07eb-4886-bd27-360cdd308357 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.783654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede68723-07eb-4886-bd27-360cdd308357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.805954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ede68723-07eb-4886-bd27-360cdd308357 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.807318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ede68723-07eb-4886-bd27-360cdd308357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.810921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 768/1546] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:02.816309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edd075-1726-4663-b4b0-a7c41048c8db None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.816309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edd075-1726-4663-b4b0-a7c41048c8db None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.816309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edd075-1726-4663-b4b0-a7c41048c8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.816842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 779/1547] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.818132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659440-dcf1-458f-a9f2-b530b19a5c75 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.818423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659440-dcf1-458f-a9f2-b530b19a5c75 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.818753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659440-dcf1-458f-a9f2-b530b19a5c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.837409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-43edd075-1726-4663-b4b0-a7c41048c8db None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.838722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3e659440-dcf1-458f-a9f2-b530b19a5c75 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.838889 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-43edd075-1726-4663-b4b0-a7c41048c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.839902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e659440-dcf1-458f-a9f2-b530b19a5c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.846972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 769/1548] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.850932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5159ce-17e0-454e-8c7c-dd9b92c71e3d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.850932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5159ce-17e0-454e-8c7c-dd9b92c71e3d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.850932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5159ce-17e0-454e-8c7c-dd9b92c71e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.851295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 780/1549] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.851295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c047-9f53-40e0-bd27-91a0369537a6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.851295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c047-9f53-40e0-bd27-91a0369537a6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.851656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c047-9f53-40e0-bd27-91a0369537a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.869881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8e5159ce-17e0-454e-8c7c-dd9b92c71e3d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.871111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e5159ce-17e0-454e-8c7c-dd9b92c71e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.873435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2519c047-9f53-40e0-bd27-91a0369537a6 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.874653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2519c047-9f53-40e0-bd27-91a0369537a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.880022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 770/1550] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.882992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea56a4d-dd2d-434d-92cb-21a701fa921f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.883319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea56a4d-dd2d-434d-92cb-21a701fa921f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.883589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea56a4d-dd2d-434d-92cb-21a701fa921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.888643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 781/1551] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.888643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c047-9f53-40e0-bd27-91a0369537a6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:02.898459 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c047-9f53-40e0-bd27-91a0369537a6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:02.903093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53310747-da9a-4c58-b1b2-3e19562c2207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:02.903718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53310747-da9a-4c58-b1b2-3e19562c2207 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.904002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53310747-da9a-4c58-b1b2-3e19562c2207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.904283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53310747-da9a-4c58-b1b2-3e19562c2207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.905098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53310747-da9a-4c58-b1b2-3e19562c2207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:02.906757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53310747-da9a-4c58-b1b2-3e19562c2207 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:02.909066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9ea56a4d-dd2d-434d-92cb-21a701fa921f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.910264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9ea56a4d-dd2d-434d-92cb-21a701fa921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.917842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 782/1552] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:02 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:02.921098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 771/1553] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.921203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e40af-66fa-4fb5-ada2-185b663fec44 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.921203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e40af-66fa-4fb5-ada2-185b663fec44 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.921362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00192f-f107-4aac-a703-4b9f6c46fbf0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.921448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e40af-66fa-4fb5-ada2-185b663fec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.921533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00192f-f107-4aac-a703-4b9f6c46fbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.921533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00192f-f107-4aac-a703-4b9f6c46fbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.943763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2e00192f-f107-4aac-a703-4b9f6c46fbf0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.943975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b35e40af-66fa-4fb5-ada2-185b663fec44 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.945033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2e00192f-f107-4aac-a703-4b9f6c46fbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.945218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b35e40af-66fa-4fb5-ada2-185b663fec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.953679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 772/1554] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.956516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00192f-f107-4aac-a703-4b9f6c46fbf0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:02.956694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 783/1555] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.956694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b10b-ff26-4871-b7aa-1fb8b650339e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.956694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b10b-ff26-4871-b7aa-1fb8b650339e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.956694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b10b-ff26-4871-b7aa-1fb8b650339e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.967386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00192f-f107-4aac-a703-4b9f6c46fbf0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:02.972408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb118-94fd-4684-a895-bb6972d401b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:02.973056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb118-94fd-4684-a895-bb6972d401b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.973353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb118-94fd-4684-a895-bb6972d401b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.973612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb118-94fd-4684-a895-bb6972d401b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.974306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb118-94fd-4684-a895-bb6972d401b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:02.974611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb118-94fd-4684-a895-bb6972d401b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:02.976031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5933b10b-ff26-4871-b7aa-1fb8b650339e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:02.976296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb118-94fd-4684-a895-bb6972d401b9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:02.977286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5933b10b-ff26-4871-b7aa-1fb8b650339e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:02.987924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 784/1556] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:02.992506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 773/1557] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:02.992632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db161d8-f775-45f7-94ca-f9d33b7ff751 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.992632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db161d8-f775-45f7-94ca-f9d33b7ff751 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.992632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db161d8-f775-45f7-94ca-f9d33b7ff751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:02.992931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5f19-af74-4a53-84ae-53bb29b15f4b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:02.992931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5f19-af74-4a53-84ae-53bb29b15f4b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:02.992931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5f19-af74-4a53-84ae-53bb29b15f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.011401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-46cf5f19-af74-4a53-84ae-53bb29b15f4b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:03.012640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-46cf5f19-af74-4a53-84ae-53bb29b15f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:03.013407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7db161d8-f775-45f7-94ca-f9d33b7ff751 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:03.014683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7db161d8-f775-45f7-94ca-f9d33b7ff751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:03.020950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 774/1558] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:03.021770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf5f19-af74-4a53-84ae-53bb29b15f4b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.023515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 785/1559] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:03.024604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db161d8-f775-45f7-94ca-f9d33b7ff751 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.032022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf5f19-af74-4a53-84ae-53bb29b15f4b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.034798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db161d8-f775-45f7-94ca-f9d33b7ff751 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.036888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.037538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.037818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.038092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.038791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.039708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.040380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.040599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.040763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.041056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.041767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:03.042078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.043099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:35:03.043099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Jul 26 15:35:03.043099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: Jul 26 15:35:03.044041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b79f22-4f0b-491d-b32d-f941c3b2148a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.052111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 775/1560] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:03 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:03.052111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.056967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 786/1561] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:03.058212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a14d5-7b11-4158-82d5-6d854f913c7f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.058510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a14d5-7b11-4158-82d5-6d854f913c7f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.058792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a14d5-7b11-4158-82d5-6d854f913c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.070630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e8c5c-f537-46a4-bee2-48a9f6c21ed3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.075366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.076037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.076326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.076649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.077352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:03.077699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.079046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-842a14d5-7b11-4158-82d5-6d854f913c7f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:03.079834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.080067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-842a14d5-7b11-4158-82d5-6d854f913c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:03.089107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 787/1562] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:03.089867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a14d5-7b11-4158-82d5-6d854f913c7f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.100797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 776/1563] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:03.100797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.101132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a14d5-7b11-4158-82d5-6d854f913c7f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.105556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.106206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.106502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.106776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.107615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:03.107950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.109602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f875-d1f4-423b-8e96-05a0061422fc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.109801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.115948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.116608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.116902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.117180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.117859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.118887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:35:03.118887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:35:03.118887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:35:03.120100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.123428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 788/1564] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:03.124413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.135204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d5dae-56bd-4bba-a943-109483f1614b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.140603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.141306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.141728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.142192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.142978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:03.143293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.144998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.157193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 789/1565] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:03.160926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.167639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b3fb-1107-4efc-bf25-f9fb794a206c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.172148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.173324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.173825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.174159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.174874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:03.175204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.176865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.191405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 790/1566] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:03.191626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.201539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641afb1-e6fd-47a1-9853-0e80421e799d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.206609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.207286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.207565 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.207881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] PATH_INFO: `/v3/projects/1165baf67df0484dbe4163c5a750784a/users/e2e9553297c842b3a3c308cdd09aad8d/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.219003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:03.224390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1165baf67df0484dbe4163c5a750784a, user_id=e2e9553297c842b3a3c308cdd09aad8d, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.227279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.240955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 777/1567] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/projects => generated 386 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:03.242011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.276139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:03.307962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5e323-0504-4f5a-a5a6-8a8b124d4ab6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.313025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.313736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.314028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.314328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.315078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.317035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.338784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 791/1568] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:03 2022] PUT /identity/v3/projects/1165baf67df0484dbe4163c5a750784a/users/e2e9553297c842b3a3c308cdd09aad8d/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:03.339931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.404076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176e48c-707e-466f-878b-263b7dce9cef admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.409597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.410509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.410920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.411204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.412246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:03.412585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.414233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.426735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 792/1569] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:03.434403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.434566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 778/1570] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/projects => generated 378 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:03.434566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.439823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6543d5d-e4a4-443f-a689-acb7e2fdc587 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.498824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.499450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.499585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccaaf3-281a-42e0-899c-6b948f86c3a0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.499735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.500012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.500939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.502919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.504975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.505612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.505924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.506417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.507088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.509026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.602139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 793/1571] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/projects => generated 394 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:03.603218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.613705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 779/1572] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/projects => generated 384 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:03.613705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.670713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327198fe-2f8f-4e8e-a614-819cdb05a250 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.676231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.676564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf271a11-d828-4d03-9a15-e49c1acf85a0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.676889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.677169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.677445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.678109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.680129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.681565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a3785-562b-4de2-a413-d082a48e0c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.682186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a3785-562b-4de2-a413-d082a48e0c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.682467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a3785-562b-4de2-a413-d082a48e0c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.682739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a3785-562b-4de2-a413-d082a48e0c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.683433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a3785-562b-4de2-a413-d082a48e0c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.685358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a3785-562b-4de2-a413-d082a48e0c8a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.695076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:03.784958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 794/1573] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/projects => generated 364 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:03.785191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 780/1574] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/projects => generated 380 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:03.785856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.786359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818281-c3ed-4718-9c08-3f41c79db0ee None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.786650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818281-c3ed-4718-9c08-3f41c79db0ee None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.786927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818281-c3ed-4718-9c08-3f41c79db0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.806820 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a0818281-c3ed-4718-9c08-3f41c79db0ee None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:03.808060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a0818281-c3ed-4718-9c08-3f41c79db0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:03.829340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:03.851686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92f35e-8237-4caa-9567-1df5200c994e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.872170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 781/1575] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:03.873142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0818281-c3ed-4718-9c08-3f41c79db0ee None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.884419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0818281-c3ed-4718-9c08-3f41c79db0ee None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.893098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.893098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.893098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.893098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.893491 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.893999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.915125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.919969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.919969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.919969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.919969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.919969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:35:03.919969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Jul 26 15:35:03.919969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: Jul 26 15:35:03.920568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:03.971534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 782/1576] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:03.972402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:03.982199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae18102-00e2-49a1-9191-964437b4f401 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:03.987746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:03.988367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:03.988676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:03.988958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:03.989623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:03.992093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.040527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 795/1577] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:04.041247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.051317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068515a8-5940-450f-88fa-9b83b0a16833 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.056007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.056658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.056945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.057220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.057884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.059812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.088382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 783/1578] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:03 2022] POST /identity/v3/projects => generated 404 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:04.089441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.143523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 796/1579] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:04 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:04.144302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.153079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8dda2-4154-48a0-9db9-c093cb0cadc1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.155670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0da6-0304-4fd4-8853-fbb8d6cc284d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.157861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.158566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.158773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.159051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.159782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.161732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.162166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.162918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.163208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.163489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.164162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.166128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.258066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 784/1580] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:04 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:04.258881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.264952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 797/1581] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:04 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:04.264952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.271495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce29db4-43c0-47ce-ba0c-0f35ed027848 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.276376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.277034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.277326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.277607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.278284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.280203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.324931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeeaf70-54be-4984-844c-777debbd1beb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.329573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.330206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.330593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.330887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.340058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.341517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.346296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:04.352443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 798/1582] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:04.354085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.363815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74017b7-43fa-45ec-a69a-3ada2d562e81 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.368521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.369119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.369421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.369685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.370386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.372116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.379363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 785/1583] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:04 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:04.381058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.388917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 799/1584] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.388917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.392196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312526bf-69eb-4a5c-af8d-1adfac32edc1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.397120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.397685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.397998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.398315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.398435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927035ac-1b42-40d3-80b7-c245421aa2d6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.399022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.400700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.403219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.403831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.404109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.404385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.405081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.408676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.412117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 786/1585] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.413423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.425376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc51629-8020-4a45-a903-860184af74d8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.430076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.430686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.430963 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.431236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.431927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.433557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.444289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 787/1586] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.445100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.454744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94240572-6eba-4b69-8c8b-e0ea40e614e9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.459356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.459958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.460241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.460579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.461414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.462911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.473538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 788/1587] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.474206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.483828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396d1cf-ffb9-4d7b-a9d2-227c1c2abc1a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.489124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.489731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.490039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.490323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.491337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.493018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.501943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 800/1588] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:04 2022] POST /identity/v3/users => generated 303 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:04.508337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.508504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 789/1589] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.508504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.515565 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439f78d-1cc5-4502-b6b5-042a35853053 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.521593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.522231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.522521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.522755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bc057-8df9-473f-91c7-4c7ed0b925d8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.522888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.523513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.525211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.528263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.528921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.529209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.529492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.530199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.532009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.540219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 801/1590] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.540219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.545054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 790/1591] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.545200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.553255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:04.553403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d740a90-bc8c-4f34-b90f-ef2279a571ce admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.558041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.558608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.558896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.559174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.559883 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.561625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.561972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e878e60-f998-4db4-b2b0-67748bec10c0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.569478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.570118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.570381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.570654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.571443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.573063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.576493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 802/1592] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.576493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.584526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 791/1593] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.584962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.586754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c21ba-c138-4140-9b70-657efec93c64 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.591673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.592350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.592675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.592960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.593769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.594927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95febdc-4b8d-4d78-9977-8f816988c58f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.595669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.599620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.600241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.600622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.600891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.601590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.603193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.610497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 803/1594] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.610497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.614393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 792/1595] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.617659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.619921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087433-915e-4eac-84f3-6c5bce7591b5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.624708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.625320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.630069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.630069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.630069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.630415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42d015-87b5-4970-8c5d-f972440b0357 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.630525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.633709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.634795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.635086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.635365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.636061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.637700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.639996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 804/1596] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.644116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.652394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248280bc-c1f0-4be2-8596-082ffc3556a0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.653203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 793/1597] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:04 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:04.654159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.659182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.659912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.660330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.660653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] PATH_INFO: `/v3/projects/f81499626e98449897622291738f07f7/users/c2e3c15511fe433a9278248c528c86e0/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.666074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cfdd0-7911-45d3-8136-cb5a4019a641 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.671274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.672045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.672400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.672860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] PATH_INFO: `/v3/projects/03338c8787e749e38b882038a546a408/users/f31c861cbc294353b79ed18658fe1236/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.683590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81499626e98449897622291738f07f7, user_id=c2e3c15511fe433a9278248c528c86e0, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.687357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.689363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=03338c8787e749e38b882038a546a408, user_id=f31c861cbc294353b79ed18658fe1236, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.690814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:35:04.690814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:35:04.690814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:35:04.693069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.777526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 805/1598] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:04 2022] PUT /identity/v3/projects/f81499626e98449897622291738f07f7/users/c2e3c15511fe433a9278248c528c86e0/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:04.778309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.800585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 794/1599] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:04 2022] PUT /identity/v3/projects/03338c8787e749e38b882038a546a408/users/f31c861cbc294353b79ed18658fe1236/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:04.801415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:04.847838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca6ac-1e2c-48b3-aa74-8b0e613ac450 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.879972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7b27c-bff1-42f4-a118-1a8178fdc03a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:04.885096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.885792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.886086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.886400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] PATH_INFO: `/v3/projects/1ae9a8b5a9ec48f392c43a71f142b102/users/494aa7af44284866b02f2a295b8b05aa/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.906731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae9a8b5a9ec48f392c43a71f142b102, user_id=494aa7af44284866b02f2a295b8b05aa, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.911795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:04.913218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:04.914086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:04.914441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:04.914792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] PATH_INFO: `/v3/projects/cc117969e9394769a471d512ce80734a/users/c2bf7ea384a142d1bc23b23ab503fca0/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:04.928938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:04.932639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc117969e9394769a471d512ce80734a, user_id=c2bf7ea384a142d1bc23b23ab503fca0, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:04.935567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.028541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 795/1600] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:04 2022] PUT /identity/v3/projects/1ae9a8b5a9ec48f392c43a71f142b102/users/494aa7af44284866b02f2a295b8b05aa/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:05.032915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.033041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 806/1601] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:04 2022] PUT /identity/v3/projects/cc117969e9394769a471d512ce80734a/users/c2bf7ea384a142d1bc23b23ab503fca0/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:05.033695 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.094706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90db44-8da3-4c1b-9b6e-80b2515d43d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.098315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935a907-daf1-4a10-867c-80923a7916ba admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.100126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093d64c-d0e8-4c9f-8aee-cc91916cdce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.100947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093d64c-d0e8-4c9f-8aee-cc91916cdce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.101259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093d64c-d0e8-4c9f-8aee-cc91916cdce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.101542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093d64c-d0e8-4c9f-8aee-cc91916cdce2 admin admin] PATH_INFO: `/v3/projects/bb91cae932274d20a4a88af94098b700/users/9030d3574dff4eddb3da85fccc075dd3/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.103300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.105634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.106578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.106908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.107708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.109452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.118264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c093d64c-d0e8-4c9f-8aee-cc91916cdce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb91cae932274d20a4a88af94098b700, user_id=9030d3574dff4eddb3da85fccc075dd3, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.120978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c093d64c-d0e8-4c9f-8aee-cc91916cdce2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.122361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 807/1602] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:05 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:05.125352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.138312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983dc35-7ec1-4df5-b11e-1dede2298e34 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.145166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe3dba-fbfc-4ac1-ab74-defa475a5d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.145835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3dba-fbfc-4ac1-ab74-defa475a5d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.146126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3dba-fbfc-4ac1-ab74-defa475a5d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.146410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3dba-fbfc-4ac1-ab74-defa475a5d89 admin admin] PATH_INFO: `/v3/projects/417b9d6cde90467ebe75c790867a8c89/users/858548bfa9b7455cb518849f31471865/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.165308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe3dba-fbfc-4ac1-ab74-defa475a5d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417b9d6cde90467ebe75c790867a8c89, user_id=858548bfa9b7455cb518849f31471865, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.166160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe3dba-fbfc-4ac1-ab74-defa475a5d89 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.201818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 796/1603] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:05 2022] PUT /identity/v3/projects/bb91cae932274d20a4a88af94098b700/users/9030d3574dff4eddb3da85fccc075dd3/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:05.201818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fa58b-d7a0-40bc-a24a-7dca505f7d78 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.201818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fa58b-d7a0-40bc-a24a-7dca505f7d78 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.201818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fa58b-d7a0-40bc-a24a-7dca505f7d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.219816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-672fa58b-d7a0-40bc-a24a-7dca505f7d78 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:05.220922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-672fa58b-d7a0-40bc-a24a-7dca505f7d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.244279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:05.253719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 808/1604] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:05 2022] PUT /identity/v3/projects/417b9d6cde90467ebe75c790867a8c89/users/858548bfa9b7455cb518849f31471865/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:05.257221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874c97-5094-4a14-97d7-8ff56f1622c9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.257221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874c97-5094-4a14-97d7-8ff56f1622c9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.257221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874c97-5094-4a14-97d7-8ff56f1622c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.290938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-24874c97-5094-4a14-97d7-8ff56f1622c9 None None] MFA Rules not processed for user `f31c861cbc294353b79ed18658fe1236`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:05.292227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-24874c97-5094-4a14-97d7-8ff56f1622c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.311389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 797/1605] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:05.312836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479b4c-25ea-4f6c-b44d-1b0991fd0e1d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.313765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479b4c-25ea-4f6c-b44d-1b0991fd0e1d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.314056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479b4c-25ea-4f6c-b44d-1b0991fd0e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.334710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d9479b4c-25ea-4f6c-b44d-1b0991fd0e1d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:05.335947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d9479b4c-25ea-4f6c-b44d-1b0991fd0e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.371835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 809/1606] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:05.373333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef75a9-567a-4c20-9e0c-bc8c2d9c4678 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.373484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef75a9-567a-4c20-9e0c-bc8c2d9c4678 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.373768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef75a9-567a-4c20-9e0c-bc8c2d9c4678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.395425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-eeef75a9-567a-4c20-9e0c-bc8c2d9c4678 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:05.396742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-eeef75a9-567a-4c20-9e0c-bc8c2d9c4678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.412328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 798/1607] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:05.413344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9479b4c-25ea-4f6c-b44d-1b0991fd0e1d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.423075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9479b4c-25ea-4f6c-b44d-1b0991fd0e1d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.428540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f57be7-c44f-4149-8737-fa12fd5ba13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.429258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f57be7-c44f-4149-8737-fa12fd5ba13d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.429541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f57be7-c44f-4149-8737-fa12fd5ba13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.429828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f57be7-c44f-4149-8737-fa12fd5ba13d admin admin] PATH_INFO: `/v3/projects/d14a8e24368f48bdbe65894851f29155/users/1768069b13a54e50b492746a4295713d/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.446918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f57be7-c44f-4149-8737-fa12fd5ba13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14a8e24368f48bdbe65894851f29155, user_id=1768069b13a54e50b492746a4295713d, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.449685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f57be7-c44f-4149-8737-fa12fd5ba13d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.455652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:05.467703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 810/1608] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:05.469010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.469299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.470134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.503952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] MFA Rules not processed for user `9030d3574dff4eddb3da85fccc075dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:05.507098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.567791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 799/1609] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:05 2022] PUT /identity/v3/projects/d14a8e24368f48bdbe65894851f29155/users/1768069b13a54e50b492746a4295713d/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:05.568454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663703c-82aa-46e6-b224-4625492efb81 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.568853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663703c-82aa-46e6-b224-4625492efb81 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.569136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663703c-82aa-46e6-b224-4625492efb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.595453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 811/1610] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:05.599035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.599035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.599209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5663703c-82aa-46e6-b224-4625492efb81 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:05.599869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5663703c-82aa-46e6-b224-4625492efb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.674638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fd22e-5e45-4cf2-b370-13591d930bf8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.691385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 800/1611] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:05.693181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272ebe-8764-4cb1-9ff6-ea71002a75df None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.693778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272ebe-8764-4cb1-9ff6-ea71002a75df None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.693977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272ebe-8764-4cb1-9ff6-ea71002a75df None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.695886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 801/1612] 10.222.0.128 () {54 vars in 967 bytes} [Tue Jul 26 15:35:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:05.696728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04272ebe-8764-4cb1-9ff6-ea71002a75df None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.736546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.737850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.738152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.738431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.741604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:05.748638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.750357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.763242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 812/1613] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:05.771162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.771459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04272ebe-8764-4cb1-9ff6-ea71002a75df None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.775647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337df5-cdad-4461-80ea-a65c0089e887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.781172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337df5-cdad-4461-80ea-a65c0089e887 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.781172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337df5-cdad-4461-80ea-a65c0089e887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.781172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337df5-cdad-4461-80ea-a65c0089e887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.788556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65e16-ac66-4902-8afc-d1dab46a33e5 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.791904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337df5-cdad-4461-80ea-a65c0089e887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.793432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.793787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337df5-cdad-4461-80ea-a65c0089e887 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.794345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.794708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.795054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.809866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.811469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.813173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 802/1614] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:05.815231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2e65-9066-434d-9fbb-b713e57e7fe0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.815783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2e65-9066-434d-9fbb-b713e57e7fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.816124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2e65-9066-434d-9fbb-b713e57e7fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.825011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 813/1615] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:05.826510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.842201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0290-9b99-4098-8837-0b4468c82e12 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.851435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.852430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.855619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.855619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.855619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.856340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bdba2e65-9066-434d-9fbb-b713e57e7fe0 None None] MFA Rules not processed for user `1768069b13a54e50b492746a4295713d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:05.857817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:05.858584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bdba2e65-9066-434d-9fbb-b713e57e7fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:05.954292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 803/1616] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:05.955189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba2e65-9066-434d-9fbb-b713e57e7fe0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:05.968204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba2e65-9066-434d-9fbb-b713e57e7fe0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:05.973199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f370-9243-41bd-b46c-b119df201fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:05.973874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f370-9243-41bd-b46c-b119df201fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:05.974161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f370-9243-41bd-b46c-b119df201fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:05.974447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f370-9243-41bd-b46c-b119df201fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:05.988688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f370-9243-41bd-b46c-b119df201fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:05.990918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f370-9243-41bd-b46c-b119df201fe4 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.004434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 804/1617] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:06.005607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7fcdf-2078-4326-ab80-919fc62469d6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.006359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7fcdf-2078-4326-ab80-919fc62469d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.006359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7fcdf-2078-4326-ab80-919fc62469d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.063891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d9e7fcdf-2078-4326-ab80-919fc62469d6 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:06.065571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d9e7fcdf-2078-4326-ab80-919fc62469d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:06.070832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 814/1618] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:05 2022] POST /identity/v3/projects => generated 364 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:06.072623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.101486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:06.248882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 805/1619] 10.222.0.128 () {58 vars in 1064 bytes} [Tue Jul 26 15:35:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:06.249887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7fcdf-2078-4326-ab80-919fc62469d6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.264850 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7fcdf-2078-4326-ab80-919fc62469d6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:06.274153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d496d0-cda0-4f7f-a7a1-28ff79d09b59 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:06.274547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:06.275366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.279598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.279598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.279598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:06.281336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:06.282584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.283146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.283592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.284020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.285932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:06.289007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.392737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 806/1620] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:06 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:06.393851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.423193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dfb3f-5e33-4aae-8570-0e66feef9543 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:06.432122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:06.432971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.433366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.434140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.448783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 815/1621] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:06 2022] POST /identity/v3/projects => generated 404 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:06.451360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.455726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:06.606300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ac8d4-4cdf-4ce9-85cb-747b8aa0da65 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:06.611597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:06.611597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.612451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:06.613190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.613546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.613895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.614683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:06.616484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.627920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 807/1622] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:06.631278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 816/1623] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:06 2022] GET /identity/v3/roles => generated 1427 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:06.634908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.640238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.662539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945462f-c9a2-49bf-9fdc-6494517e0d15 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:06.666135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbf2e2-7e99-48f3-9acd-347dd9a97341 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:06.676118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:06.676410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:06.676789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.677067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.677343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.677452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.677708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.678001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] PATH_INFO: `/v3/projects/2802c7cd7089467b9918c68aafc37287/users/cec39020751443328ad74ad0914ae630/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.678142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:06.679938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.710276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2802c7cd7089467b9918c68aafc37287, user_id=cec39020751443328ad74ad0914ae630, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:06.725259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.833216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 808/1624] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:06 2022] POST /identity/v3/users => generated 309 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:06.834381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.840087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 817/1625] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:06 2022] PUT /identity/v3/projects/2802c7cd7089467b9918c68aafc37287/users/cec39020751443328ad74ad0914ae630/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:06.843505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:06.854051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb3e01-c113-4fb0-b392-16993c9a7ad2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:06.974371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d94ee-528a-40c4-9220-a145432000a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:06.975203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d94ee-528a-40c4-9220-a145432000a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.975664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d94ee-528a-40c4-9220-a145432000a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.975991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d94ee-528a-40c4-9220-a145432000a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.976920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d94ee-528a-40c4-9220-a145432000a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:06.978690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d94ee-528a-40c4-9220-a145432000a1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:06.993876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 809/1626] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:06 2022] GET /identity/v3/roles => generated 1427 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:06.996177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b077b0-3e30-4c49-b6e7-aaaf284085a6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:06.996406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b077b0-3e30-4c49-b6e7-aaaf284085a6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:06.996860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b077b0-3e30-4c49-b6e7-aaaf284085a6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:06.999449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 810/1627] 10.222.0.128 () {54 vars in 917 bytes} [Tue Jul 26 15:35:06 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:07.011948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b077b0-3e30-4c49-b6e7-aaaf284085a6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:07.015531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b077b0-3e30-4c49-b6e7-aaaf284085a6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.023552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.024445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.024879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.025425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] PATH_INFO: `/v3/projects/d038a65394184f779935caba4ace95d1/users/89a5dcfd8bd8453dafc0d199b580b870/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.062866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d038a65394184f779935caba4ace95d1, user_id=89a5dcfd8bd8453dafc0d199b580b870, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.066929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.078661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f32a4-746a-4e01-a3e4-9f19f10da926 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.084253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728d03-89c3-4709-adaa-b83bd87f5f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.084986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728d03-89c3-4709-adaa-b83bd87f5f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.085278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728d03-89c3-4709-adaa-b83bd87f5f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.085559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728d03-89c3-4709-adaa-b83bd87f5f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.086268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728d03-89c3-4709-adaa-b83bd87f5f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.087923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728d03-89c3-4709-adaa-b83bd87f5f6d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.094100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:07.100024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 818/1628] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:06 2022] GET /identity/v3/roles => generated 1427 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:07.101481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bd03-c116-4719-95b6-fe185c1db47b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.101883 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bd03-c116-4719-95b6-fe185c1db47b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.102269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bd03-c116-4719-95b6-fe185c1db47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.123858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-90f8bd03-c116-4719-95b6-fe185c1db47b None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:07.126374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90f8bd03-c116-4719-95b6-fe185c1db47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:07.173915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 811/1629] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:06 2022] PUT /identity/v3/projects/d038a65394184f779935caba4ace95d1/users/89a5dcfd8bd8453dafc0d199b580b870/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:07.175637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:07.225849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 819/1630] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:07.232867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8bd03-c116-4719-95b6-fe185c1db47b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:07.246273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8bd03-c116-4719-95b6-fe185c1db47b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.253830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.256133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.256133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.256133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.256133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.257767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.257978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afddaa6-183a-48ab-ab63-10d041215d48 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.263381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.264059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.264351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.264885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] PATH_INFO: `/v3/projects/2802c7cd7089467b9918c68aafc37287/users/cec39020751443328ad74ad0914ae630/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.273536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2802c7cd7089467b9918c68aafc37287, user_id=cec39020751443328ad74ad0914ae630, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.280777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 820/1631] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:07 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:07.280777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:07.281002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.286242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:07.403096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 812/1632] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:07 2022] PUT /identity/v3/projects/2802c7cd7089467b9918c68aafc37287/users/cec39020751443328ad74ad0914ae630/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:07.403096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:07.404169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837f7f-53af-49b8-8afb-7290409c61c3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.407281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67455d1-1b53-4787-a67f-7e7dfea43989 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.413097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1037f-99ab-44a7-b164-98bffcdba393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.413849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1037f-99ab-44a7-b164-98bffcdba393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.414540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1037f-99ab-44a7-b164-98bffcdba393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.414717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1037f-99ab-44a7-b164-98bffcdba393 admin admin] PATH_INFO: `/v3/projects/d038a65394184f779935caba4ace95d1/users/89a5dcfd8bd8453dafc0d199b580b870/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.418443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1037f-99ab-44a7-b164-98bffcdba393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d038a65394184f779935caba4ace95d1, user_id=89a5dcfd8bd8453dafc0d199b580b870, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.420099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1037f-99ab-44a7-b164-98bffcdba393 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.445965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 813/1633] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:07 2022] PUT /identity/v3/projects/d038a65394184f779935caba4ace95d1/users/89a5dcfd8bd8453dafc0d199b580b870/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:07.447323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40420-93bf-4349-9d43-1f9f414df35d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.447622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40420-93bf-4349-9d43-1f9f414df35d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.447899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40420-93bf-4349-9d43-1f9f414df35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.493111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.493111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.493111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.493111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.494700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6da40420-93bf-4349-9d43-1f9f414df35d None None] MFA Rules not processed for user `cec39020751443328ad74ad0914ae630`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:07.496112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6da40420-93bf-4349-9d43-1f9f414df35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:07.588992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.591883 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.595199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 814/1634] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:07.596671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536474d-5233-4fef-8bfa-8b26d988fb5e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.596966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536474d-5233-4fef-8bfa-8b26d988fb5e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.597415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536474d-5233-4fef-8bfa-8b26d988fb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.615959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 821/1635] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:07.617473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:07.632452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c0019-4c7e-4401-9a64-c849185f3498 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.640877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0536474d-5233-4fef-8bfa-8b26d988fb5e None None] MFA Rules not processed for user `89a5dcfd8bd8453dafc0d199b580b870`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:07.645265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0536474d-5233-4fef-8bfa-8b26d988fb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:07.723460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.724384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.724614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.724899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.742258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.743953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.761272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 815/1636] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:07.767593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 822/1637] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:07.851160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:07.943440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536474d-5233-4fef-8bfa-8b26d988fb5e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:07.956449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536474d-5233-4fef-8bfa-8b26d988fb5e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:07.962286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:07.967497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:07.967497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:07.967497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:07.976988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:07.978550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:07.991512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 816/1638] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:07 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:08.092150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:08.298019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:08.310129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff909c6-ef4b-4e1d-8715-58c8759cd4fb service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:08.316235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19a454-0521-4bd6-b809-8e1cd2a935f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:08.316833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19a454-0521-4bd6-b809-8e1cd2a935f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:08.317120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19a454-0521-4bd6-b809-8e1cd2a935f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:08.317404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19a454-0521-4bd6-b809-8e1cd2a935f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:08.420150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19a454-0521-4bd6-b809-8e1cd2a935f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:08.423802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19a454-0521-4bd6-b809-8e1cd2a935f8 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:08.449257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 823/1639] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:09.165654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:09.317243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af932c-bd60-44cc-8670-59893628407c service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:09.331376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:09.333209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:09.334204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:09.335080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] PATH_INFO: `/v3/users/1768069b13a54e50b492746a4295713d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:09.343785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1768069b13a54e50b492746a4295713d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:09.354751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:35:09.354751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jul 26 15:35:09.354751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: Jul 26 15:35:09.364301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:09.655390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d356c0-ca08-4715-b990-f8ba49bd4571 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:09.655874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d356c0-ca08-4715-b990-f8ba49bd4571 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:09.655954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d356c0-ca08-4715-b990-f8ba49bd4571 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:09.658928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 824/1640] 10.222.0.128 () {54 vars in 967 bytes} [Tue Jul 26 15:35:09 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:35:09.667516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186f75-3ead-438b-b71a-c507f882cf4c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:09.667810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186f75-3ead-438b-b71a-c507f882cf4c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:09.668116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186f75-3ead-438b-b71a-c507f882cf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:09.695105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5c186f75-3ead-438b-b71a-c507f882cf4c None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:09.696757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5c186f75-3ead-438b-b71a-c507f882cf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:09.785618 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1768069b13a54e50b492746a4295713d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:09.787877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 825/1641] 10.222.0.128 () {58 vars in 1064 bytes} [Tue Jul 26 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:09.796808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdf792-2d42-4205-a65f-1fb8c448df89 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:09.798019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdf792-2d42-4205-a65f-1fb8c448df89 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:09.798815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdf792-2d42-4205-a65f-1fb8c448df89 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:09.809732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 826/1642] 10.222.0.128 () {54 vars in 917 bytes} [Tue Jul 26 15:35:09 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:09.811271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:09.811271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:09.811986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:09.815501 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1768069b13a54e50b492746a4295713d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:09.816352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1768069b13a54e50b492746a4295713d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:09.845509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:09.846935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:09.859026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 827/1643] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:09.879157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:09.881455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:09.900210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:09.904666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a6146-fbaf-4783-917d-667991c6b2fe None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:09.909408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c356669-df9e-4e91-95ef-71d9c816ec64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:09.910591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c356669-df9e-4e91-95ef-71d9c816ec64 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:09.910591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c356669-df9e-4e91-95ef-71d9c816ec64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:09.910807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c356669-df9e-4e91-95ef-71d9c816ec64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:09.922057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c356669-df9e-4e91-95ef-71d9c816ec64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:09.924499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c356669-df9e-4e91-95ef-71d9c816ec64 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:09.937035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 828/1644] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:09.949243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1768069b13a54e50b492746a4295713d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:09.949935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1768069b13a54e50b492746a4295713d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:09.994896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 817/1645] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:09 2022] DELETE /identity/v3/users/1768069b13a54e50b492746a4295713d => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:09.999833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:10.000408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:10.001219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:10.042949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:10.044183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:10.158863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 829/1646] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:10.176923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:10.178390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:10.255493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61518ee-fb98-4306-b685-0363eb816097 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:10.259882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:10.260818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:10.261098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:10.261570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:10.275872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:10.277818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:10.305176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 818/1647] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:12.151593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:12.153173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:12.176498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07090037-9a8f-4260-b4dd-b71ab0bcfaa3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:12.191936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:12.193015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:12.193456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:12.193886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] PATH_INFO: `/v3/projects/d14a8e24368f48bdbe65894851f29155` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:12.197508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14a8e24368f48bdbe65894851f29155)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:12.198832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:35:12.198832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Jul 26 15:35:12.198832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: Jul 26 15:35:12.201279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:12.433862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14a8e24368f48bdbe65894851f29155', 'request_id': 'req-27a4de70-5ace-4f0e-ba74-a24d3f101e81'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:12.465184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14a8e24368f48bdbe65894851f29155', 'request_id': 'req-27a4de70-5ace-4f0e-ba74-a24d3f101e81'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:12.499827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14a8e24368f48bdbe65894851f29155', 'request_id': 'req-27a4de70-5ace-4f0e-ba74-a24d3f101e81'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:12.600554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] The token cache is being invalidate because project d14a8e24368f48bdbe65894851f29155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:12.602882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:12.603347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:12.608014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 830/1648] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:12 2022] DELETE /identity/v3/projects/d14a8e24368f48bdbe65894851f29155 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:12.612127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:12.630171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13920-29ba-4cd6-a0e8-dd978c455860 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:12.655856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:12.838568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:12.839249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:12.839537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:12.839819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] PATH_INFO: `/v3/users/89a5dcfd8bd8453dafc0d199b580b870` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:12.841155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a5dcfd8bd8453dafc0d199b580b870)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:12.843234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:13.121307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a5dcfd8bd8453dafc0d199b580b870'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:13.136587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a5dcfd8bd8453dafc0d199b580b870'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:13.142427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a5dcfd8bd8453dafc0d199b580b870'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:13.269558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a5dcfd8bd8453dafc0d199b580b870'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:13.272817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b4323c55-132e-454d-bfe5-95ae50449b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a5dcfd8bd8453dafc0d199b580b870'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:13.334831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 819/1649] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:12 2022] DELETE /identity/v3/users/89a5dcfd8bd8453dafc0d199b580b870 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:15.052434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:15.055793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:15.119494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4de70-5ace-4f0e-ba74-a24d3f101e81 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:15.308328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:15.309282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:15.313144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:15.313144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] PATH_INFO: `/v3/projects/d038a65394184f779935caba4ace95d1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:15.313144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d038a65394184f779935caba4ace95d1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:15.319821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:15.363179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:15.495694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd038a65394184f779935caba4ace95d1', 'request_id': 'req-3fa78331-efe1-4114-8e20-68efd2f6c52c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:15.511560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd038a65394184f779935caba4ace95d1', 'request_id': 'req-3fa78331-efe1-4114-8e20-68efd2f6c52c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:15.549380 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd038a65394184f779935caba4ace95d1', 'request_id': 'req-3fa78331-efe1-4114-8e20-68efd2f6c52c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:15.744788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] The token cache is being invalidate because project d038a65394184f779935caba4ace95d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:15.745399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:15.746076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3fa78331-efe1-4114-8e20-68efd2f6c52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:15.748239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 831/1650] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:15 2022] DELETE /identity/v3/projects/d038a65394184f779935caba4ace95d1 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:15.754587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7faf5-6e1f-4463-a93b-c33c023365fa None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:15.754587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7faf5-6e1f-4463-a93b-c33c023365fa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:15.754587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7faf5-6e1f-4463-a93b-c33c023365fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:15.792804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-25f7faf5-6e1f-4463-a93b-c33c023365fa None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:15.798420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-25f7faf5-6e1f-4463-a93b-c33c023365fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:15.954369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 820/1651] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:15.964803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa37131-3a44-4cf5-a747-e1f7ec862452 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:15.965388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa37131-3a44-4cf5-a747-e1f7ec862452 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:15.965871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa37131-3a44-4cf5-a747-e1f7ec862452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.026143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-faa37131-3a44-4cf5-a747-e1f7ec862452 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:16.026961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-faa37131-3a44-4cf5-a747-e1f7ec862452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:16.042760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 832/1652] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:16.047150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7faf5-6e1f-4463-a93b-c33c023365fa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:16.066846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7faf5-6e1f-4463-a93b-c33c023365fa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:16.107640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:16.108287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:16.108601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:16.108898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.110102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:16.110406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:16.112038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:16.143113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 821/1653] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:16.146181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa37131-3a44-4cf5-a747-e1f7ec862452 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:16.168578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa37131-3a44-4cf5-a747-e1f7ec862452 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:16.182579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:16.183146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:16.183441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:16.183718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.184613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:16.186791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:16.300197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 833/1654] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:16 2022] POST /identity/v3/projects => generated 404 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:16.304703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:16.319036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d443d5-53ec-41e8-b1a8-1f6b5579e704 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:16.352344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:16.479693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:16.480907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:16.480907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:16.481263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.482275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:16.487069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:16.584777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 822/1655] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:16 2022] POST /identity/v3/users => generated 303 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:16.588357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:16.616925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4861-5736-4d0a-9e3d-07cb29643f04 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:16.623790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:16.624640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:16.625764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:16.626192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.628836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:16.634830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:16.653853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 834/1656] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:16 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:16.660346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:16.671320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd407421-5e27-4a2a-87b0-8a17ff380105 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:16.686380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fa10a-fd1c-4865-8111-35103e993b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:16.687037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fa10a-fd1c-4865-8111-35103e993b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:16.687422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fa10a-fd1c-4865-8111-35103e993b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:16.688285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fa10a-fd1c-4865-8111-35103e993b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.690591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fa10a-fd1c-4865-8111-35103e993b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:16.695866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fa10a-fd1c-4865-8111-35103e993b78 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:16.717983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 823/1657] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:16 2022] GET /identity/v3/roles => generated 1427 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:16.719593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:16.739583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b477fc-eff5-41d7-8fee-dbe9c2b57e79 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:16.749675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:16.750696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:16.751321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:16.751825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] PATH_INFO: `/v3/projects/646ded2b58a149bebe7872ed35ef5643/users/61f93a142db4439abff3256bdd5c48ce/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.780927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=646ded2b58a149bebe7872ed35ef5643, user_id=61f93a142db4439abff3256bdd5c48ce, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:16.787916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:16.881635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 835/1658] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:16 2022] PUT /identity/v3/projects/646ded2b58a149bebe7872ed35ef5643/users/61f93a142db4439abff3256bdd5c48ce/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:16.887410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:16.887910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:16.888010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:16.914112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:16.938790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:17.163855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 824/1659] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:17.270465 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:17.274104 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:17.528170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97b68-1c2c-4aa9-a467-26f4510eebdd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:17.547883 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ccb4f-9ce3-44fb-95f5-49f96134abe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:17.548587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ccb4f-9ce3-44fb-95f5-49f96134abe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:17.548875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ccb4f-9ce3-44fb-95f5-49f96134abe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:17.549156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ccb4f-9ce3-44fb-95f5-49f96134abe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:17.587522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ccb4f-9ce3-44fb-95f5-49f96134abe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:17.591182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ccb4f-9ce3-44fb-95f5-49f96134abe8 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:17.646295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 836/1660] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:21.309643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:21.311960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:21.365221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee509a-81bd-4f6f-a434-cf29d6f5bbba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:21.385073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:21.386034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:21.386323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:21.386600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] PATH_INFO: `/v3/users/f31c861cbc294353b79ed18658fe1236` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:21.388969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31c861cbc294353b79ed18658fe1236)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:21.393575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:21.688646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31c861cbc294353b79ed18658fe1236'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:21.709877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31c861cbc294353b79ed18658fe1236'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:21.711638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31c861cbc294353b79ed18658fe1236'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:21.895943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31c861cbc294353b79ed18658fe1236'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:21.896764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08595fc7-a2df-4697-b84d-ac4375b9d6ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31c861cbc294353b79ed18658fe1236'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:21.965221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 825/1661] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:21 2022] DELETE /identity/v3/users/f31c861cbc294353b79ed18658fe1236 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:21.967099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1562-ce06-43e8-9ba1-1fa7625b5efd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:21.967727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1562-ce06-43e8-9ba1-1fa7625b5efd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:21.967727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1562-ce06-43e8-9ba1-1fa7625b5efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.037516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:22.042154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:22.042154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.072669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4dfa1562-ce06-43e8-9ba1-1fa7625b5efd None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:22.078341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4dfa1562-ce06-43e8-9ba1-1fa7625b5efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:22.098335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:22.118716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] MFA Rules not processed for user `c2bf7ea384a142d1bc23b23ab503fca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:22.119979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:22.204446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:22.255248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 837/1662] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:22.255248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2060-4525-47ff-8ffb-a32346bece57 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:22.255248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2060-4525-47ff-8ffb-a32346bece57 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:22.255248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2060-4525-47ff-8ffb-a32346bece57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.323512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-761b2060-4525-47ff-8ffb-a32346bece57 None None] MFA Rules not processed for user `61f93a142db4439abff3256bdd5c48ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:22.323512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-761b2060-4525-47ff-8ffb-a32346bece57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:22.363686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 826/1663] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:22.364778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:22.366158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:22.495176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 838/1664] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:22.495176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:22.495176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:22.495176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.537396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d6e90-3d97-4907-b456-8b9a6d512b92 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:22.538197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] MFA Rules not processed for user `c2bf7ea384a142d1bc23b23ab503fca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:22.539439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:22.541566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:22.548557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:22.548557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:22.548557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.558372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 839/1665] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:22.559462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:22.561189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:22.565370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:22.571833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:22.590671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 827/1666] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:22.592549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:22.595921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:22.605587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138fafb-db56-4bdc-b5ca-f2dbf89f40f9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:22.617703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:22.617992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:22.618281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:22.618564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.619416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:22.621409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:22.761432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bfc3a-b46b-4fa7-85b3-2194430db77a service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:22.763929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 840/1667] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:22 2022] POST /identity/v3/projects => generated 406 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:22.768263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:22.919965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cae24-1278-48a9-b40c-2167b882ce18 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:22.929411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f276e4b-dc20-4d8b-ac9a-4d19cf5d66bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:22.930057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f276e4b-dc20-4d8b-ac9a-4d19cf5d66bb service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:22.930351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f276e4b-dc20-4d8b-ac9a-4d19cf5d66bb service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:22.930632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f276e4b-dc20-4d8b-ac9a-4d19cf5d66bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.938581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:22.939381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:22.939785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:22.940353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:22.941146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:22.947439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:23.068419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f276e4b-dc20-4d8b-ac9a-4d19cf5d66bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:23.069617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f276e4b-dc20-4d8b-ac9a-4d19cf5d66bb service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:23.085815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 828/1668] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:23.087187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:23.089575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:23.089575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:23.151079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 841/1669] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:22 2022] POST /identity/v3/users => generated 310 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:35:23.154637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:23.163114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] MFA Rules not processed for user `e2e9553297c842b3a3c308cdd09aad8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:23.164649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:23.182667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea30c7-fa51-4f2b-a79a-baaf640662ce admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:23.211173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:23.212069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:23.212407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:23.212822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:23.213660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:23.215592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:23.259420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 842/1670] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:23 2022] GET /identity/v3/roles => generated 1427 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:35:23.264153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:23.287255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7bfbd-d2c0-4d23-8ce9-529e4f1ebd98 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:23.304566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:23.307520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:23.309173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:23.310431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] PATH_INFO: `/v3/projects/f8ed9e07c5284fdc80dcd22dd275ac80/users/39450def45cd49cc84738b51d6efb0c2/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:23.353856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ed9e07c5284fdc80dcd22dd275ac80, user_id=39450def45cd49cc84738b51d6efb0c2, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:23.356944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:23.397495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 829/1671] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:23.409809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:23.411478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:23.449751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3bf4e-9b7d-4771-90c8-7d36e43ee0ed None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:23.474981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:23.475877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:23.476322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:23.476997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:23.504851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:23.508643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:23.528309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 843/1672] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:23 2022] PUT /identity/v3/projects/f8ed9e07c5284fdc80dcd22dd275ac80/users/39450def45cd49cc84738b51d6efb0c2/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:23.530717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:23.538318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 830/1673] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:35:23.703724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949d6a6-7036-4d42-b600-0ead57283fb8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:23.710902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:23.712034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:23.712545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:23.713075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:23.714289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:23.716268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:23.737711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 844/1674] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:23 2022] GET /identity/v3/roles => generated 1427 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:35:23.739228 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:23.755153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f902555-1f38-4423-86a4-41402cc30046 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:23.767378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4ace7-5796-43a9-8a4c-1e3d2ac2cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:23.768682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4ace7-5796-43a9-8a4c-1e3d2ac2cd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:23.769100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4ace7-5796-43a9-8a4c-1e3d2ac2cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:23.769452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4ace7-5796-43a9-8a4c-1e3d2ac2cd7c admin admin] PATH_INFO: `/v3/projects/f8ed9e07c5284fdc80dcd22dd275ac80/users/39450def45cd49cc84738b51d6efb0c2/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:23.822752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4ace7-5796-43a9-8a4c-1e3d2ac2cd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ed9e07c5284fdc80dcd22dd275ac80, user_id=39450def45cd49cc84738b51d6efb0c2, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:23.828278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4ace7-5796-43a9-8a4c-1e3d2ac2cd7c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:23.872963 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 831/1675] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:23 2022] PUT /identity/v3/projects/f8ed9e07c5284fdc80dcd22dd275ac80/users/39450def45cd49cc84738b51d6efb0c2/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:24.130945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:24.455634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:24.460286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:24.477842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48963d-f482-48e1-a863-0154fdc25db4 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:24.478329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48963d-f482-48e1-a863-0154fdc25db4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:24.479051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48963d-f482-48e1-a863-0154fdc25db4 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:24.482525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 832/1676] 10.222.0.128 () {54 vars in 917 bytes} [Tue Jul 26 15:35:24 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:35:24.503039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:24.517392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe0f7-a6d4-4ae7-a361-472491305ba7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:24.517392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe0f7-a6d4-4ae7-a361-472491305ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:24.517392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe0f7-a6d4-4ae7-a361-472491305ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:24.534496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a152f25-9b46-4690-9fde-e2c71cc68971 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:24.628870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-840fe0f7-a6d4-4ae7-a361-472491305ba7 None None] MFA Rules not processed for user `c6ecc741022f4d6189f4d3f19b4cdbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:24.630183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-840fe0f7-a6d4-4ae7-a361-472491305ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:24.750309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:24.765682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:24.765682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:24.765682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] PATH_INFO: `/v3/users/9030d3574dff4eddb3da85fccc075dd3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:24.765682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9030d3574dff4eddb3da85fccc075dd3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:24.765682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:24.883383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 833/1677] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:35:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:24.886245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e144e61-8370-4334-8064-71762f89a19e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:24.889332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e144e61-8370-4334-8064-71762f89a19e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:24.892123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e144e61-8370-4334-8064-71762f89a19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:24.906298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:24.968648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6e144e61-8370-4334-8064-71762f89a19e None None] MFA Rules not processed for user `494aa7af44284866b02f2a295b8b05aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:24.970695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e144e61-8370-4334-8064-71762f89a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:25.107912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9030d3574dff4eddb3da85fccc075dd3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.110476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9030d3574dff4eddb3da85fccc075dd3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.129644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9030d3574dff4eddb3da85fccc075dd3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.158470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 834/1678] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:25.158470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33814539-f23e-4d35-9265-228381c0765e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:25.158470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33814539-f23e-4d35-9265-228381c0765e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:25.158470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33814539-f23e-4d35-9265-228381c0765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:25.182160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9030d3574dff4eddb3da85fccc075dd3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.220696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-33814539-f23e-4d35-9265-228381c0765e None None] MFA Rules not processed for user `c2e3c15511fe433a9278248c528c86e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:25.222474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-33814539-f23e-4d35-9265-228381c0765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:25.320612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9030d3574dff4eddb3da85fccc075dd3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.341006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 845/1679] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:24 2022] DELETE /identity/v3/users/9030d3574dff4eddb3da85fccc075dd3 => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:25.344033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:25.348915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:25.357967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 835/1680] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:25.362161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33814539-f23e-4d35-9265-228381c0765e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:25.367332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-33814539-f23e-4d35-9265-228381c0765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:25.408548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33814539-f23e-4d35-9265-228381c0765e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:25.551536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd87da1-d420-4fed-8efd-007482793f5f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:25.573166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:25.573832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:25.574122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:25.574400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:25.576525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:25.577711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:25.579423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:25.579835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] PATH_INFO: `/v3/projects/03338c8787e749e38b882038a546a408` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:25.580237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:25.590989 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=03338c8787e749e38b882038a546a408)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:25.606406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:25.606506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:25.610000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:25.633700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 846/1681] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:25.636183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:25.640629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:25.763872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea62f65-22b1-46ed-aa32-a5c0730dac6d service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:25.799001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03338c8787e749e38b882038a546a408', 'request_id': 'req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.844738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a4355-d325-4f00-abce-1467d99d2d2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:25.845398 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a4355-d325-4f00-abce-1467d99d2d2d service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:25.845861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a4355-d325-4f00-abce-1467d99d2d2d service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:25.846292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a4355-d325-4f00-abce-1467d99d2d2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:25.851834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c24a4355-d325-4f00-abce-1467d99d2d2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:25.930209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03338c8787e749e38b882038a546a408', 'request_id': 'req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.939781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03338c8787e749e38b882038a546a408', 'request_id': 'req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:25.970175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a4355-d325-4f00-abce-1467d99d2d2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:25.972072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a4355-d325-4f00-abce-1467d99d2d2d service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:25.994688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] The token cache is being invalidate because project 03338c8787e749e38b882038a546a408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:25.995062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:26.002178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-954fc3dd-14ca-41e7-ad7b-67a3138b4dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:26.009182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 847/1682] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:26.014218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 836/1683] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:25 2022] DELETE /identity/v3/projects/03338c8787e749e38b882038a546a408 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:26.014390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.014390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.014390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.016968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94fa9f-411f-4204-8800-a1a6dfdf855d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.017622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94fa9f-411f-4204-8800-a1a6dfdf855d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.017907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94fa9f-411f-4204-8800-a1a6dfdf855d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.061387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:26.072693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:26.076889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:26.091973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3a94fa9f-411f-4204-8800-a1a6dfdf855d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:26.095790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3a94fa9f-411f-4204-8800-a1a6dfdf855d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:26.257072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 848/1684] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:26.289617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:26.292129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:26.331220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 837/1685] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:26.339014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.339298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.345367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.381591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:26.387231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:26.408605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 838/1686] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:26.408605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:26.408605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:26.435148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ed6f1-6f1d-481a-88b6-61e387788267 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:26.443288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:26.444271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.444704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.445074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.467627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a547a2-830d-4ac6-ab25-b25162b50d3a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:26.467857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:26.475955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:26.478966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.479456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.479746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.519314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:26.524046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:26.555153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 849/1687] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:35:26.556040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:26.570521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa774-cedc-483d-b2c6-32cbb0b39b17 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:26.576774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:26.578566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.579120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.579717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.580641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:26.581427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:26.586267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:26.602315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 850/1688] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:26.606902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:26.635802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bd714-ddf4-4bdd-9312-b40961c1b2e5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:26.649499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b658b-7b2c-4abe-bfaf-4fab37fa8048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:26.653664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b658b-7b2c-4abe-bfaf-4fab37fa8048 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.654185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b658b-7b2c-4abe-bfaf-4fab37fa8048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.654642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b658b-7b2c-4abe-bfaf-4fab37fa8048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.657247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b658b-7b2c-4abe-bfaf-4fab37fa8048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:26.663804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b658b-7b2c-4abe-bfaf-4fab37fa8048 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:26.676289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:26.678327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:26.690032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 839/1689] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:26.691205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:26.829436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 851/1690] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:26 2022] POST /identity/v3/projects => generated 356 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:35:26.833619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:26.847534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae903f7-493e-446c-9668-ddbb002b5257 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:26.989257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:26.990024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:26.990581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:26.990985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:26.991905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:26.994649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:27.143504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39569376-a37a-4986-9488-2e5e5f5ee995 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.143848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39569376-a37a-4986-9488-2e5e5f5ee995 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.144067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39569376-a37a-4986-9488-2e5e5f5ee995 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.146197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 852/1691] 10.222.0.128 () {54 vars in 966 bytes} [Tue Jul 26 15:35:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:27.154110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caebc6e-766e-4567-9fba-6dbd6821974b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.154395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caebc6e-766e-4567-9fba-6dbd6821974b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.154593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caebc6e-766e-4567-9fba-6dbd6821974b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.172961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 840/1692] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:26 2022] POST /identity/v3/users => generated 279 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:27.174120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:27.196835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ad333-fc95-43bb-8692-1b81a3341818 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:27.222037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4caebc6e-766e-4567-9fba-6dbd6821974b None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:27.231379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4caebc6e-766e-4567-9fba-6dbd6821974b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:27.236833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:27.237412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.237701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.238172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.250765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:27.253602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:27.272852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 841/1693] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:27 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:27.276418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:27.335449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c0b2-89b8-419c-bec3-29c8e60f937c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:27.345707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78e5d3-8eef-4618-bc45-7ccc6b6e8e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:27.345910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e5d3-8eef-4618-bc45-7ccc6b6e8e19 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.346097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e5d3-8eef-4618-bc45-7ccc6b6e8e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.346860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e5d3-8eef-4618-bc45-7ccc6b6e8e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.347095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78e5d3-8eef-4618-bc45-7ccc6b6e8e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:27.350538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78e5d3-8eef-4618-bc45-7ccc6b6e8e19 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:27.365582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:27.366377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 842/1694] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:27 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:27.367795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b7cdf-6401-4b12-899e-e753c5bcbb12 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.369753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b7cdf-6401-4b12-899e-e753c5bcbb12 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.370049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b7cdf-6401-4b12-899e-e753c5bcbb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.479021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cf2b7cdf-6401-4b12-899e-e753c5bcbb12 None None] MFA Rules not processed for user `858548bfa9b7455cb518849f31471865`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:27.482772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 853/1695] 10.222.0.128 () {58 vars in 1063 bytes} [Tue Jul 26 15:35:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:27.486410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caebc6e-766e-4567-9fba-6dbd6821974b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:27.490244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf2b7cdf-6401-4b12-899e-e753c5bcbb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:27.512328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caebc6e-766e-4567-9fba-6dbd6821974b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:27.518612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ca7fe-170e-4bc1-9cde-966b32fa6a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:27.519740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ca7fe-170e-4bc1-9cde-966b32fa6a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.520311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ca7fe-170e-4bc1-9cde-966b32fa6a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.520837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ca7fe-170e-4bc1-9cde-966b32fa6a5f admin admin] PATH_INFO: `/v3/projects/db52330537c74e4bb90ec6a055d97b7f/users/d411a7966fec4d0e86d69999f816c13d/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.551121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ca7fe-170e-4bc1-9cde-966b32fa6a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=db52330537c74e4bb90ec6a055d97b7f, user_id=d411a7966fec4d0e86d69999f816c13d, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:27.559185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ca7fe-170e-4bc1-9cde-966b32fa6a5f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:27.651650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 854/1696] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:27 2022] PUT /identity/v3/projects/db52330537c74e4bb90ec6a055d97b7f/users/d411a7966fec4d0e86d69999f816c13d/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:27.661586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d66a3-39c5-4c4d-9ffc-9c242144edf8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.661586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d66a3-39c5-4c4d-9ffc-9c242144edf8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.661586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d66a3-39c5-4c4d-9ffc-9c242144edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.730418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d79d66a3-39c5-4c4d-9ffc-9c242144edf8 None None] MFA Rules not processed for user `c2bf7ea384a142d1bc23b23ab503fca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:27.731759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d79d66a3-39c5-4c4d-9ffc-9c242144edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:27.754514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 843/1697] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:27.755132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b7cdf-6401-4b12-899e-e753c5bcbb12 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:27.773852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b7cdf-6401-4b12-899e-e753c5bcbb12 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:27.932790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec77eb-8d65-45e3-bf3a-5400e5ceb0fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:27.933346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec77eb-8d65-45e3-bf3a-5400e5ceb0fa service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.933881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec77eb-8d65-45e3-bf3a-5400e5ceb0fa service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.934256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec77eb-8d65-45e3-bf3a-5400e5ceb0fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.942250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4fec77eb-8d65-45e3-bf3a-5400e5ceb0fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:27.948008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 855/1698] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:27.949681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eee64b-cb45-47d6-9f25-5958739581c4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.969047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eee64b-cb45-47d6-9f25-5958739581c4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.969368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eee64b-cb45-47d6-9f25-5958739581c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:27.969835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec77eb-8d65-45e3-bf3a-5400e5ceb0fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:27.975484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec77eb-8d65-45e3-bf3a-5400e5ceb0fa service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:27.992073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 844/1699] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:27.993713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a89c-3820-49af-9838-1e3f7c345062 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:27.994160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a89c-3820-49af-9838-1e3f7c345062 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:27.994635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a89c-3820-49af-9838-1e3f7c345062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:28.028770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a720a89c-3820-49af-9838-1e3f7c345062 None None] MFA Rules not processed for user `858548bfa9b7455cb518849f31471865`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:28.028770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a720a89c-3820-49af-9838-1e3f7c345062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:28.078666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a4eee64b-cb45-47d6-9f25-5958739581c4 None None] MFA Rules not processed for user `d411a7966fec4d0e86d69999f816c13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:28.080940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a4eee64b-cb45-47d6-9f25-5958739581c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:28.159482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:28.174637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 845/1700] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:28.176174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650754b-443b-424f-9ec2-ebd8f4ada2cd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:28.176597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650754b-443b-424f-9ec2-ebd8f4ada2cd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:28.177072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650754b-443b-424f-9ec2-ebd8f4ada2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:28.241405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6650754b-443b-424f-9ec2-ebd8f4ada2cd None None] MFA Rules not processed for user `39450def45cd49cc84738b51d6efb0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:28.246246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6650754b-443b-424f-9ec2-ebd8f4ada2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:28.286375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 856/1701] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:28.287297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eee64b-cb45-47d6-9f25-5958739581c4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:28.493318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eee64b-cb45-47d6-9f25-5958739581c4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:28.500733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:28.505198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:28.505198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:28.505198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:28.514546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:28.517364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:28.543824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 857/1702] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:28.543824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:28.558887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 846/1703] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:28.642885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:28.766258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676ed3-747b-4d9f-892c-1d127c1ab39b service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:28.805603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:28.807328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:28.807712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:28.808208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:28.809108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:28.816602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:28.891172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:29.055053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 858/1704] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:28 2022] POST /identity/v3/projects => generated 356 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:29.056115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650754b-443b-424f-9ec2-ebd8f4ada2cd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:29.081049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650754b-443b-424f-9ec2-ebd8f4ada2cd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:29.300363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:29.305103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:29.305402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:29.305683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:29.306379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:29.308332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:29.443601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 847/1705] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:29 2022] POST /identity/v3/users => generated 285 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:29.444483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:29.474847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6624bc-1be4-4701-a18d-f4e23ff21660 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:29.512160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:29.512847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:29.513140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:29.513423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:29.514155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:29.524504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:29.561426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 859/1706] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:29 2022] GET /identity/v3/roles => generated 1427 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:29.562344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:29.576427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fea6c-68b3-4d51-b748-370bf2f92ad4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:29.598564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:29.599678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:29.602628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:29.604883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:29.606177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] PATH_INFO: `/v3/projects/820d413fc6014edc92a710f001a02aca/users/8b165af8cccd4e4884adae421fedfa61/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:29.668571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820d413fc6014edc92a710f001a02aca, user_id=8b165af8cccd4e4884adae421fedfa61, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:29.685250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:29.800114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 848/1707] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:29 2022] PUT /identity/v3/projects/820d413fc6014edc92a710f001a02aca/users/8b165af8cccd4e4884adae421fedfa61/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:29.804227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:29.821331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353b607-65ce-446f-a472-8a532e5894b5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:29.919880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:29.923122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:29.996483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:29.997132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:29.997422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:29.997712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:29.998434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:30.000250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:30.025432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:30.038311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 860/1708] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:29 2022] GET /identity/v3/roles => generated 1427 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:30.048449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:30.083359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b400e-2540-41f1-8630-b02fcc0e7ad3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:30.103214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01266154-6031-495a-a032-4cfbaab3c6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:30.104149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266154-6031-495a-a032-4cfbaab3c6f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:30.104733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266154-6031-495a-a032-4cfbaab3c6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:30.105225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266154-6031-495a-a032-4cfbaab3c6f7 admin admin] PATH_INFO: `/v3/projects/820d413fc6014edc92a710f001a02aca/users/8b165af8cccd4e4884adae421fedfa61/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:30.117530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01266154-6031-495a-a032-4cfbaab3c6f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820d413fc6014edc92a710f001a02aca, user_id=8b165af8cccd4e4884adae421fedfa61, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:30.122702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01266154-6031-495a-a032-4cfbaab3c6f7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:30.150087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e02e-551c-4588-a5c5-2eedfa9309d0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:30.173796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 861/1709] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:30 2022] PUT /identity/v3/projects/820d413fc6014edc92a710f001a02aca/users/8b165af8cccd4e4884adae421fedfa61/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:30.175348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07060f5-0e83-4b54-af5d-18b33519c7ce None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:30.175646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07060f5-0e83-4b54-af5d-18b33519c7ce None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:30.175934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07060f5-0e83-4b54-af5d-18b33519c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:30.261427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c07060f5-0e83-4b54-af5d-18b33519c7ce None None] MFA Rules not processed for user `8b165af8cccd4e4884adae421fedfa61`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:30.263044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c07060f5-0e83-4b54-af5d-18b33519c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:30.364138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:30.370227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:30.370548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:30.370837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] PATH_INFO: `/v3/projects/bb91cae932274d20a4a88af94098b700` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:30.380913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb91cae932274d20a4a88af94098b700)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:30.388006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:30.493249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 862/1710] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:30.496368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5b92-6310-44dc-9ccb-aa91f5652715 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:30.497110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5b92-6310-44dc-9ccb-aa91f5652715 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:30.497570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5b92-6310-44dc-9ccb-aa91f5652715 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:30.509372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 863/1711] 10.222.0.128 () {54 vars in 917 bytes} [Tue Jul 26 15:35:30 2022] GET /identity => generated 271 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:30.513583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f5b92-6310-44dc-9ccb-aa91f5652715 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:30.514341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-241f5b92-6310-44dc-9ccb-aa91f5652715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:30.606677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb91cae932274d20a4a88af94098b700', 'request_id': 'req-8fd054a7-7023-45e0-b994-ece5ad230ba1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:30.675888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:30.685706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f5b92-6310-44dc-9ccb-aa91f5652715 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:30.707297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb91cae932274d20a4a88af94098b700', 'request_id': 'req-8fd054a7-7023-45e0-b994-ece5ad230ba1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:30.715699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb91cae932274d20a4a88af94098b700', 'request_id': 'req-8fd054a7-7023-45e0-b994-ece5ad230ba1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:30.743069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] The token cache is being invalidate because project bb91cae932274d20a4a88af94098b700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:30.743446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:30.744338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8fd054a7-7023-45e0-b994-ece5ad230ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:30.756308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 849/1712] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:29 2022] DELETE /identity/v3/projects/bb91cae932274d20a4a88af94098b700 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:30.757743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:30.758037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:30.761743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:30.828131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] MFA Rules not processed for user `c6ecc741022f4d6189f4d3f19b4cdbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:30.830015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:30.845194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:30.847052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:30.847429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:30.854209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:30.855206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:30.936007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:30.961665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 850/1713] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:35:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:30.962819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:30.965070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:31.038285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:31.040538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:31.101019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 864/1714] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:31.226028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df525f64-16b1-4655-9ca8-79e17002620d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:31.245980 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:31.246766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:31.247841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:31.247841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] PATH_INFO: `/v3/users/cec39020751443328ad74ad0914ae630` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:31.255284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec39020751443328ad74ad0914ae630)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:31.258245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:31.472990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:31.476977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:31.515728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640db70-a8b5-4eb7-b31b-3be5f0d254ad service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:31.526394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:31.528057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:31.529485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:31.530883 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:31.538982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:31.606820 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec39020751443328ad74ad0914ae630'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:31.617841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec39020751443328ad74ad0914ae630'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:31.631335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec39020751443328ad74ad0914ae630'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:31.666942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:31.669807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:31.699903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 865/1715] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:31.700843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:31.707944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:31.745768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60754-6ba6-4589-ae84-51f482433a54 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:31.770499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec39020751443328ad74ad0914ae630'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:31.774066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e33791-c22a-4362-9b3a-5f11b7fc9f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:31.774711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e33791-c22a-4362-9b3a-5f11b7fc9f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:31.774993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e33791-c22a-4362-9b3a-5f11b7fc9f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:31.775269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e33791-c22a-4362-9b3a-5f11b7fc9f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:31.778746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d2782c15-163b-49b5-b8cb-df0bda9dfa90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec39020751443328ad74ad0914ae630'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:31.789062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-62e33791-c22a-4362-9b3a-5f11b7fc9f25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:31.824179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e33791-c22a-4362-9b3a-5f11b7fc9f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:31.827027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e33791-c22a-4362-9b3a-5f11b7fc9f25 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:31.857670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 866/1716] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:31.884165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:31.908761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 851/1717] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:30 2022] DELETE /identity/v3/users/cec39020751443328ad74ad0914ae630 => generated 0 bytes in 942 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:32.320910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e7cd6-194a-4a58-89d6-3def03c2aa32 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:32.321539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e7cd6-194a-4a58-89d6-3def03c2aa32 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:32.322077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e7cd6-194a-4a58-89d6-3def03c2aa32 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:32.323876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 867/1718] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:35:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:32.414347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:32.415344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:32.416427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:32.481691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:32.489408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:32.561777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f87ff-5e73-45c7-86c1-843c0d0d6e51 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:32.562324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f87ff-5e73-45c7-86c1-843c0d0d6e51 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:32.562974 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f87ff-5e73-45c7-86c1-843c0d0d6e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:32.615609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-089f87ff-5e73-45c7-86c1-843c0d0d6e51 None None] MFA Rules not processed for user `e2e9553297c842b3a3c308cdd09aad8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:32.619373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 852/1719] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:32.620170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-089f87ff-5e73-45c7-86c1-843c0d0d6e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:32.653938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:32.657364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:32.677634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ba759-0085-4944-8266-df18fb1a8ebd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:32.699573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:32.700509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:32.700941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:32.701388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:32.739412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:32.741361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:32.766351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 853/1720] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:35:32.770391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 868/1721] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:33.033464 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:33.802022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:33.805714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:33.884959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a554875-5b23-41e2-90b9-d349ae1cb7ed None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:33.885661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a554875-5b23-41e2-90b9-d349ae1cb7ed None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:33.885997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a554875-5b23-41e2-90b9-d349ae1cb7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:34.020204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6322ca1-eb38-49fe-afd2-df5ba28a14cd service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:34.049240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9a554875-5b23-41e2-90b9-d349ae1cb7ed None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:34.049521 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:34.050934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9a554875-5b23-41e2-90b9-d349ae1cb7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:34.051742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:34.060454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:34.061464 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] PATH_INFO: `/v3/projects/2802c7cd7089467b9918c68aafc37287` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:34.065944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2802c7cd7089467b9918c68aafc37287)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:34.082466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:34.261846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 869/1722] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:35:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:34.314545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2802c7cd7089467b9918c68aafc37287', 'request_id': 'req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:34.430148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2802c7cd7089467b9918c68aafc37287', 'request_id': 'req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:34.442539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2802c7cd7089467b9918c68aafc37287', 'request_id': 'req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:34.506251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] The token cache is being invalidate because project 2802c7cd7089467b9918c68aafc37287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:34.506507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:34.514380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5fb3b602-a4e0-417b-90f1-f8d9c4a1ded8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:34.521042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 854/1723] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:33 2022] DELETE /identity/v3/projects/2802c7cd7089467b9918c68aafc37287 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:34.526261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:34.526628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:34.528713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:34.591059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:34.600532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:34.723751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:34.783772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 870/1724] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:34.793252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e589f-021a-49b0-aa7c-63bedb9821fa None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:34.793860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e589f-021a-49b0-aa7c-63bedb9821fa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:34.794323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e589f-021a-49b0-aa7c-63bedb9821fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:34.821623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:34.823201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:34.823887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cb5e589f-021a-49b0-aa7c-63bedb9821fa None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:34.825580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cb5e589f-021a-49b0-aa7c-63bedb9821fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:34.865702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 855/1725] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:34.868858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e589f-021a-49b0-aa7c-63bedb9821fa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:34.887016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e589f-021a-49b0-aa7c-63bedb9821fa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:34.904352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:34.905990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:34.906445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:34.906866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:34.907757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:34.908222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:34.910414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:34.949419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 856/1726] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:34.949419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:34.961802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871339-14f0-41ae-997f-cc0df983d8b0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:34.974544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:34.975444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:34.975892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:34.976171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:34.976908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:34.978876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:35.021885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ffb6-1cc4-4b31-a7db-1c211e14db2e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:35.021885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:35.022530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:35.022530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:35.023821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:35.025445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:35.143812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 857/1727] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:34 2022] POST /identity/v3/projects => generated 384 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:35.148798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:35.350716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:35.361210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:35.399684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c297424-a193-4605-b36d-79933b61bd2e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:35.400727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 871/1728] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:35:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:35:35.411221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:35.412201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:35.412705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:35.413161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:35.414031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:35.421114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:35.517559 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:35.609443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 858/1729] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:35 2022] POST /identity/v3/users => generated 293 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:35:35.614743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:35.651302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142f03-af3e-4c27-aec4-d053e7664770 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:35.672685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:35.674490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:35.674776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:35.675268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:35.678267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:35.681082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:35.712287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 872/1730] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:35 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:35.717286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:35.730714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79527f-c809-45fa-b94b-6ba3f9596422 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:35.758435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea3da-7e7a-46f6-94a7-b5cc966acd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:35.759343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea3da-7e7a-46f6-94a7-b5cc966acd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:35.759775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea3da-7e7a-46f6-94a7-b5cc966acd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:35.760227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea3da-7e7a-46f6-94a7-b5cc966acd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:35.761154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea3da-7e7a-46f6-94a7-b5cc966acd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:35.764384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea3da-7e7a-46f6-94a7-b5cc966acd22 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:35.813939 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 859/1731] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:35 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:35.816217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:35.843411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0563-d16b-4ad1-85a3-ccbec2c5fe22 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:35.857116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:35.857816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:35.859226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:35.859226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] PATH_INFO: `/v3/projects/48924a41afc743139fc284a0842f0ced/users/4dafe1221f43425aa5863eca8ac149c4/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:35.922864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48924a41afc743139fc284a0842f0ced, user_id=4dafe1221f43425aa5863eca8ac149c4, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:35.925799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:36.096940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 873/1732] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:35 2022] PUT /identity/v3/projects/48924a41afc743139fc284a0842f0ced/users/4dafe1221f43425aa5863eca8ac149c4/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:36.109675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48569b1b-e713-4500-b253-fc98e73b1894 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:36.110210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48569b1b-e713-4500-b253-fc98e73b1894 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:36.110402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48569b1b-e713-4500-b253-fc98e73b1894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:36.180631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-48569b1b-e713-4500-b253-fc98e73b1894 None None] MFA Rules not processed for user `4dafe1221f43425aa5863eca8ac149c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:36.181876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-48569b1b-e713-4500-b253-fc98e73b1894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:36.382415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 860/1733] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:36.386932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:36.487645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:36.535644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee7ddd-5b16-4240-8c85-25d87cbbc06a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:36.542355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:36.543264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:36.543905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:36.545963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:36.548154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:36.551506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:36.721969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 874/1734] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:36 2022] POST /identity/v3/projects => generated 384 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:36.724229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48569b1b-e713-4500-b253-fc98e73b1894 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:36.737989 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48569b1b-e713-4500-b253-fc98e73b1894 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:36.887827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:36.888925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:36.890078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:36.890586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:36.891485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:36.894280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:37.023455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 861/1735] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:36 2022] POST /identity/v3/users => generated 299 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:37.028396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:37.039553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb296bfd-ed28-4208-b79a-532740ab5cdd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:37.065022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418daa3-434f-4548-ac77-223ef0a42b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:37.070687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418daa3-434f-4548-ac77-223ef0a42b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.070687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418daa3-434f-4548-ac77-223ef0a42b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.070687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418daa3-434f-4548-ac77-223ef0a42b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:37.070687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418daa3-434f-4548-ac77-223ef0a42b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:37.075583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418daa3-434f-4548-ac77-223ef0a42b1e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:37.115983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:37.117129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 875/1736] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:37 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:37.134672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5280-3732-485c-99f7-3d5312d9d8ae admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:37.196351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e8c81-9038-419a-9906-4c18bb07666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:37.198877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e8c81-9038-419a-9906-4c18bb07666c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.199598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e8c81-9038-419a-9906-4c18bb07666c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.200923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e8c81-9038-419a-9906-4c18bb07666c admin admin] PATH_INFO: `/v3/projects/4a732d61db0a44bea2406e49c4226397/users/cef51e846d5e41d3af6ba63519d07392/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:37.253986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cbad4-ac96-4c94-a5dd-68293a16830c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.254262 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cbad4-ac96-4c94-a5dd-68293a16830c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.254657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cbad4-ac96-4c94-a5dd-68293a16830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:37.269410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e8c81-9038-419a-9906-4c18bb07666c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a732d61db0a44bea2406e49c4226397, user_id=cef51e846d5e41d3af6ba63519d07392, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:37.279539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e8c81-9038-419a-9906-4c18bb07666c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:37.386860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a02cbad4-ac96-4c94-a5dd-68293a16830c None None] MFA Rules not processed for user `494aa7af44284866b02f2a295b8b05aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:37.386860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a02cbad4-ac96-4c94-a5dd-68293a16830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:37.516358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 876/1737] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:37.527853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f516ee-7962-48b1-80e9-d2c195b797e9 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.527853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f516ee-7962-48b1-80e9-d2c195b797e9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.527853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f516ee-7962-48b1-80e9-d2c195b797e9 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:37.527853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 877/1738] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:35:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:37.627990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 862/1739] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:37 2022] PUT /identity/v3/projects/4a732d61db0a44bea2406e49c4226397/users/cef51e846d5e41d3af6ba63519d07392/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:37.632729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f516ee-7962-48b1-80e9-d2c195b797e9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:37.694514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33597c-2efc-453a-96c0-7c87c7f153c4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.695387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33597c-2efc-453a-96c0-7c87c7f153c4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.695976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33597c-2efc-453a-96c0-7c87c7f153c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:37.739076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7a33597c-2efc-453a-96c0-7c87c7f153c4 None None] MFA Rules not processed for user `16e8966ae76242898bc996cd039dbe31`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:37.742302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a33597c-2efc-453a-96c0-7c87c7f153c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:37.817738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f516ee-7962-48b1-80e9-d2c195b797e9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:37.832325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:37.834217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.834518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.840914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:37.846587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:37.850356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:37.890400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 878/1740] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:37 2022] GET /identity/v3/roles => generated 1427 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:37.891566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:37.924778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc5c80-c75f-4629-b3ff-ad1fb15be983 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:37.933029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:37.938880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.938880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.938880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] PATH_INFO: `/v3/projects/4a732d61db0a44bea2406e49c4226397/users/cef51e846d5e41d3af6ba63519d07392/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:37.939181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 863/1741] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:37.939263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a732d61db0a44bea2406e49c4226397, user_id=cef51e846d5e41d3af6ba63519d07392, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:37.950604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:37.992284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:37.992607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:37.992885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:38.007566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 879/1742] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:37 2022] PUT /identity/v3/projects/4a732d61db0a44bea2406e49c4226397/users/cef51e846d5e41d3af6ba63519d07392/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:38.042267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:38.044428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:38.062884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] MFA Rules not processed for user `cef51e846d5e41d3af6ba63519d07392`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:38.064978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:38.202422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:38.289141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633247ca-bd51-49f7-b98a-48ef1cde4195 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:38.303701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:38.309118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:38.319522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:38.320017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:38.320391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] PATH_INFO: `/v3/users/61f93a142db4439abff3256bdd5c48ce` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:38.335540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f93a142db4439abff3256bdd5c48ce)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:38.340281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 864/1743] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:38.340281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:38.340281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:38.351241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:38.556190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcd09-cb02-4760-87e4-d3933c87df10 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:38.577842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:38.579723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:38.580084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:38.580837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:38.621493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:38.661035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:38.696862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f93a142db4439abff3256bdd5c48ce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:38.699590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f93a142db4439abff3256bdd5c48ce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:38.710610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 865/1744] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:38.720808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:38.722780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:38.749167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f93a142db4439abff3256bdd5c48ce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:38.820026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f93a142db4439abff3256bdd5c48ce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:38.932134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e048a-f6c1-4f70-b857-38e94d053e51 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:38.935358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-388bb280-1822-4ea4-9d1e-29e36b44783f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f93a142db4439abff3256bdd5c48ce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:38.947368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 880/1745] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:38 2022] DELETE /identity/v3/users/61f93a142db4439abff3256bdd5c48ce => generated 0 bytes in 920 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:38.949191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5fe72-be25-420e-a53b-025fd557f55d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:38.949488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5fe72-be25-420e-a53b-025fd557f55d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:38.949775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5fe72-be25-420e-a53b-025fd557f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:38.954009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:38.956271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:38.958691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:38.959114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:38.965994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:39.037625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c8c5fe72-be25-420e-a53b-025fd557f55d None None] MFA Rules not processed for user `c2e3c15511fe433a9278248c528c86e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:39.049383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c8c5fe72-be25-420e-a53b-025fd557f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:39.145272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 881/1746] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:39.191989 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:39.202528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:39.239060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 866/1747] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:39.586571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:40.575397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:40.575945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:40.575945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:40.649635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] MFA Rules not processed for user `c6ecc741022f4d6189f4d3f19b4cdbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:40.656051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:40.874796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 882/1748] 10.222.0.48 () {58 vars in 1019 bytes} [Tue Jul 26 15:35:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:40.953791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:40.956315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:41.133212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d810df-2051-4537-99f7-61cef0345e63 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:41.141670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:41.142400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:41.142692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:41.142978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:41.161948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:41.163835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:41.194011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 867/1749] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:41.489874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:41.489874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:41.495248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:41.643491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7900e8-7e25-485f-b1fc-1b844a8c31e6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:41.653080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:41.656146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:41.656887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:41.657818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] PATH_INFO: `/v3/projects/646ded2b58a149bebe7872ed35ef5643` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:41.661207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] RBAC: Authorizing `identity:delete_project(project_id=646ded2b58a149bebe7872ed35ef5643)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:41.665816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:41.881708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646ded2b58a149bebe7872ed35ef5643', 'request_id': 'req-1c8b1863-1969-4846-98e8-09eb3ffcc64c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:41.921786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646ded2b58a149bebe7872ed35ef5643', 'request_id': 'req-1c8b1863-1969-4846-98e8-09eb3ffcc64c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:41.945320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646ded2b58a149bebe7872ed35ef5643', 'request_id': 'req-1c8b1863-1969-4846-98e8-09eb3ffcc64c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:42.043483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] The token cache is being invalidate because project 646ded2b58a149bebe7872ed35ef5643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:42.043783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:42.044742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:42.050348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 883/1750] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:41 2022] DELETE /identity/v3/projects/646ded2b58a149bebe7872ed35ef5643 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:42.053222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:42.073562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cf636-79f2-4b2e-aeab-d46d2c7d98a2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:42.285655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:42.287017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:42.289139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:42.290009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] PATH_INFO: `/v3/users/39450def45cd49cc84738b51d6efb0c2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:42.293140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39450def45cd49cc84738b51d6efb0c2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:42.296061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:42.389485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:42.661040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39450def45cd49cc84738b51d6efb0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:42.672591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39450def45cd49cc84738b51d6efb0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:42.674353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39450def45cd49cc84738b51d6efb0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:43.226616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39450def45cd49cc84738b51d6efb0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:43.227583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d7a78f8b-3c87-4ca2-82f8-d3741e98e610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39450def45cd49cc84738b51d6efb0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:43.289564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 868/1751] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:42 2022] DELETE /identity/v3/users/39450def45cd49cc84738b51d6efb0c2 => generated 0 bytes in 1237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:44.741859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:44.744292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:44.770232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1863-1969-4846-98e8-09eb3ffcc64c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:45.033198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:45.034031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:45.034391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:45.034757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] PATH_INFO: `/v3/projects/f8ed9e07c5284fdc80dcd22dd275ac80` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:45.037306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ed9e07c5284fdc80dcd22dd275ac80)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:45.044064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:45.252975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:45.275512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ed9e07c5284fdc80dcd22dd275ac80', 'request_id': 'req-e1d855ae-ebe1-40ac-9133-1f5125656638'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:45.305960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ed9e07c5284fdc80dcd22dd275ac80', 'request_id': 'req-e1d855ae-ebe1-40ac-9133-1f5125656638'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:45.354759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ed9e07c5284fdc80dcd22dd275ac80', 'request_id': 'req-e1d855ae-ebe1-40ac-9133-1f5125656638'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:45.476231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] The token cache is being invalidate because project f8ed9e07c5284fdc80dcd22dd275ac80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:45.476867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:45.480628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e1d855ae-ebe1-40ac-9133-1f5125656638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:45.490016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 884/1752] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:44 2022] DELETE /identity/v3/projects/f8ed9e07c5284fdc80dcd22dd275ac80 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:45.495397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71502cd5-af91-4180-b842-73b92b7dfedd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:45.495978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71502cd5-af91-4180-b842-73b92b7dfedd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:45.496080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71502cd5-af91-4180-b842-73b92b7dfedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:45.561691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-71502cd5-af91-4180-b842-73b92b7dfedd None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:45.563169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71502cd5-af91-4180-b842-73b92b7dfedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:45.681640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6c64-1e6f-4a93-8343-97e41e2dca2b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:45.681932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6c64-1e6f-4a93-8343-97e41e2dca2b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:45.682244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6c64-1e6f-4a93-8343-97e41e2dca2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:45.683082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 869/1753] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:45.771594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dedd6c64-1e6f-4a93-8343-97e41e2dca2b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:45.777961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dedd6c64-1e6f-4a93-8343-97e41e2dca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:45.809065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 885/1754] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:45.816314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71502cd5-af91-4180-b842-73b92b7dfedd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:45.865872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71502cd5-af91-4180-b842-73b92b7dfedd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:45.891335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:45.892016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:45.908074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:45.908535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:45.909192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:45.909519 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:45.926297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:45.994246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd6c64-1e6f-4a93-8343-97e41e2dca2b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:45.996673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 870/1755] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:46.064774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd6c64-1e6f-4a93-8343-97e41e2dca2b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:46.084602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996ecd9-30de-4e1d-a977-690f37b2dc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:46.086366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996ecd9-30de-4e1d-a977-690f37b2dc9e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:46.086959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996ecd9-30de-4e1d-a977-690f37b2dc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:46.100051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996ecd9-30de-4e1d-a977-690f37b2dc9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:46.102289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996ecd9-30de-4e1d-a977-690f37b2dc9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:46.114747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996ecd9-30de-4e1d-a977-690f37b2dc9e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:46.313490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 886/1756] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:45 2022] POST /identity/v3/projects => generated 372 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:46.317614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:46.349026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d3288-5dd5-461c-93a8-5cfdcb5c7eb2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:46.428458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f967-1135-41fd-9cba-1834419bea8c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:46.428773 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f967-1135-41fd-9cba-1834419bea8c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:46.429053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f967-1135-41fd-9cba-1834419bea8c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:46.430769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 887/1757] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:35:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 15:35:46.522599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e2a4-82d3-4fa9-b708-d65b611bbfbe None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:46.522945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e2a4-82d3-4fa9-b708-d65b611bbfbe None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:46.523234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e2a4-82d3-4fa9-b708-d65b611bbfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:46.588721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f614e2a4-82d3-4fa9-b708-d65b611bbfbe None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:46.589988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f614e2a4-82d3-4fa9-b708-d65b611bbfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:46.659125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:46.666930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:46.667513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:46.668657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:46.670694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:46.682893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:46.834318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 888/1758] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:35:46.916507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 871/1759] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:46 2022] POST /identity/v3/users => generated 287 bytes in 599 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:46.920659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614e2a4-82d3-4fa9-b708-d65b611bbfbe None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:46.932558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614e2a4-82d3-4fa9-b708-d65b611bbfbe None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:46.939807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:46.940471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:46.945903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:46.945903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:46.945903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:46.945903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:46.959730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 889/1760] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:46 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:46.961040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:46.974826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a33c6-1212-4744-a953-731b05e8c604 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:46.982977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898b76d-d924-44eb-86fe-636288078d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:46.986936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898b76d-d924-44eb-86fe-636288078d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:46.987933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898b76d-d924-44eb-86fe-636288078d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:46.988425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898b76d-d924-44eb-86fe-636288078d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:46.989596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898b76d-d924-44eb-86fe-636288078d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:46.991667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898b76d-d924-44eb-86fe-636288078d9d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:47.065081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:47.067289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 872/1761] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:46 2022] GET /identity/v3/roles => generated 1427 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:47.105685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ecaef-2f0e-44ed-aa0d-5403bd18715f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:47.130519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:47.130519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:47.130519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:47.130519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] PATH_INFO: `/v3/projects/0c186c1ba3bb4c9f9681c7c04ed20a03/users/aad4757e1c5548a28531307a90351c00/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:47.202368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c186c1ba3bb4c9f9681c7c04ed20a03, user_id=aad4757e1c5548a28531307a90351c00, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:47.209864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:47.366234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 890/1762] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:47 2022] PUT /identity/v3/projects/0c186c1ba3bb4c9f9681c7c04ed20a03/users/aad4757e1c5548a28531307a90351c00/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:47.377658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29989a39-7fac-48df-a60e-58600b3cd06b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:47.378226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29989a39-7fac-48df-a60e-58600b3cd06b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:47.378507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29989a39-7fac-48df-a60e-58600b3cd06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:47.388290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:47.471042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-29989a39-7fac-48df-a60e-58600b3cd06b None None] MFA Rules not processed for user `aad4757e1c5548a28531307a90351c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:47.472626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-29989a39-7fac-48df-a60e-58600b3cd06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:47.548787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:47.550771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:47.651013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 873/1763] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:47.659347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29989a39-7fac-48df-a60e-58600b3cd06b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:47.695353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b9718-cbf5-4891-b4c2-f0ce92d6542b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:47.706623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:47.709834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:47.709834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:47.709834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:47.710673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:47.729816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:47.808429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29989a39-7fac-48df-a60e-58600b3cd06b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:47.816489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff35518-4f42-4c8e-9ae7-4f664f7a75e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:47.821735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff35518-4f42-4c8e-9ae7-4f664f7a75e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:47.822336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff35518-4f42-4c8e-9ae7-4f664f7a75e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:47.822970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff35518-4f42-4c8e-9ae7-4f664f7a75e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:47.824153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff35518-4f42-4c8e-9ae7-4f664f7a75e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:47.827776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff35518-4f42-4c8e-9ae7-4f664f7a75e6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:47.870613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:47.874311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:47.897997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 891/1764] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:35:47.918025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:47.918025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:48.038045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 874/1765] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:47 2022] POST /identity/v3/projects => generated 372 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:48.040021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cd083-cd81-42a3-88f7-7e546c7d23b9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:48.041606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cd083-cd81-42a3-88f7-7e546c7d23b9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:48.042785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cd083-cd81-42a3-88f7-7e546c7d23b9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:48.048090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 875/1766] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:35:48 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:35:48.051282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cd083-cd81-42a3-88f7-7e546c7d23b9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:48.085129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6f00-aab0-405c-91ad-a6626a2cfa09 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:48.202102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253975e-1927-416a-aae0-1b11ecb72a32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:48.203245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253975e-1927-416a-aae0-1b11ecb72a32 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:48.204322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253975e-1927-416a-aae0-1b11ecb72a32 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:48.204866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253975e-1927-416a-aae0-1b11ecb72a32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:48.221115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253975e-1927-416a-aae0-1b11ecb72a32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:48.223190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253975e-1927-416a-aae0-1b11ecb72a32 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:48.238225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 892/1767] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:48.250691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b163ca-c377-47e8-9d6c-f724fbdda513 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:48.251130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b163ca-c377-47e8-9d6c-f724fbdda513 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:48.251444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b163ca-c377-47e8-9d6c-f724fbdda513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:48.309043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-00b163ca-c377-47e8-9d6c-f724fbdda513 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:48.313939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-00b163ca-c377-47e8-9d6c-f724fbdda513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:48.354753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 893/1768] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:35:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:48.435941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cd083-cd81-42a3-88f7-7e546c7d23b9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:48.463397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:48.464449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:48.464793 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:48.465074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:48.466122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:48.470398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:48.571493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 876/1769] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:48 2022] POST /identity/v3/users => generated 293 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:35:48.576760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b163ca-c377-47e8-9d6c-f724fbdda513 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:48.597659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b163ca-c377-47e8-9d6c-f724fbdda513 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:48.624413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:48.633246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:48.633683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:48.634103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:48.636016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:48.641664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:48.692660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 894/1770] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:48 2022] GET /identity/v3/roles => generated 1427 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:35:48.692950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:48.706825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f551ae2-c55c-4898-8ee5-6220feb75d29 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:48.721847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:48.722716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:48.723012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:48.723300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] PATH_INFO: `/v3/projects/379a187a97cd4a6eafcd478b2fd4972d/users/7ff20911913546648108cd057e43fd1e/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:48.742922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:48.760978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=379a187a97cd4a6eafcd478b2fd4972d, user_id=7ff20911913546648108cd057e43fd1e, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:48.767741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:48.850301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:48.852624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:48.874689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7020b-317d-4a91-85fe-4fbe4da66c85 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:48.882974 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:48.883666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:48.884039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:48.884379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:48.901874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 877/1771] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:48 2022] PUT /identity/v3/projects/379a187a97cd4a6eafcd478b2fd4972d/users/7ff20911913546648108cd057e43fd1e/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:48.902720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:48.930044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:48.936773 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:48.979720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 895/1772] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:35:49.051463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdcb33-a7c0-4753-8553-565209a4c8fb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:49.082207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a54f7-d83c-4c79-bc68-52a6ee566bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:49.083311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a54f7-d83c-4c79-bc68-52a6ee566bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:49.084715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a54f7-d83c-4c79-bc68-52a6ee566bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:49.085146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a54f7-d83c-4c79-bc68-52a6ee566bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:49.089007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a54f7-d83c-4c79-bc68-52a6ee566bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:49.093075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a54f7-d83c-4c79-bc68-52a6ee566bf3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:49.140825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 878/1773] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:48 2022] GET /identity/v3/roles => generated 1427 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:35:49.144888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:49.171147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aade2-a763-449a-b669-662e5ea7fa29 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:49.192152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:49.196659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:49.196990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:49.197247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] PATH_INFO: `/v3/projects/379a187a97cd4a6eafcd478b2fd4972d/users/7ff20911913546648108cd057e43fd1e/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:49.208393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] RBAC: Authorizing `identity:create_grant(project_id=379a187a97cd4a6eafcd478b2fd4972d, user_id=7ff20911913546648108cd057e43fd1e, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:49.223684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:49.271046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:49.301652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 896/1774] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:49 2022] PUT /identity/v3/projects/379a187a97cd4a6eafcd478b2fd4972d/users/7ff20911913546648108cd057e43fd1e/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:49.314154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e70f-0184-4264-89a0-1fb0a6ac706f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:49.314311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e70f-0184-4264-89a0-1fb0a6ac706f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:49.314575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e70f-0184-4264-89a0-1fb0a6ac706f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:49.444899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-55d1e70f-0184-4264-89a0-1fb0a6ac706f None None] MFA Rules not processed for user `7ff20911913546648108cd057e43fd1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:49.446253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55d1e70f-0184-4264-89a0-1fb0a6ac706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:49.711510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 879/1775] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:49.725009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:49.726598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:49.950085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3b29-6fe6-4e21-bebf-bc0ed9f0209b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:49.962168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:49.978324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:49.979717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:49.981934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:50.037135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:50.041488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:50.063624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 897/1776] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:50.448639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1e70f-0184-4264-89a0-1fb0a6ac706f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:50.479536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1e70f-0184-4264-89a0-1fb0a6ac706f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:50.486999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad38b-66d4-4e4a-ac05-1cf1a8438f15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:50.488287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad38b-66d4-4e4a-ac05-1cf1a8438f15 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:50.488836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad38b-66d4-4e4a-ac05-1cf1a8438f15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:50.489350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad38b-66d4-4e4a-ac05-1cf1a8438f15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:50.681400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad38b-66d4-4e4a-ac05-1cf1a8438f15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:50.683026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad38b-66d4-4e4a-ac05-1cf1a8438f15 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:50.699928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 880/1777] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:55.638372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:55.641698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:55.742415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83539760-fd4a-46bc-a8f4-c6a2cc186f49 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:55.766197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:55.767168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:55.767563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:55.767948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] PATH_INFO: `/v3/users/4dafe1221f43425aa5863eca8ac149c4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:55.769816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dafe1221f43425aa5863eca8ac149c4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:55.772051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:55.947190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dafe1221f43425aa5863eca8ac149c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:55.947768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dafe1221f43425aa5863eca8ac149c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:55.955025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dafe1221f43425aa5863eca8ac149c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:55.968537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dafe1221f43425aa5863eca8ac149c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:56.053816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dafe1221f43425aa5863eca8ac149c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:56.060391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 898/1778] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:55 2022] DELETE /identity/v3/users/4dafe1221f43425aa5863eca8ac149c4 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:56.065015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:56.065517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:56.065635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:56.073667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:56.085713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:56.086936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:56.146810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 881/1779] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:56.157436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:56.158920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:56.168753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:56.217462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd26a45-fe4b-4d66-956c-d1fbde68a2b8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:56.221102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:56.221728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:56.222016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:56.222298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:56.231308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:56.232887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:56.242719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 899/1780] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:56.775018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:56.776768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:56.791348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dec12-42ee-4e43-b456-bb11139fdcc8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:56.798769 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:56.799732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:56.800262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:56.801986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] PATH_INFO: `/v3/projects/48924a41afc743139fc284a0842f0ced` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:56.803500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=48924a41afc743139fc284a0842f0ced)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:56.806673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:56.887730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:56.889479 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:56.911282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b241a-be9a-49c5-ac89-2cf9e2bcc87b service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:56.938353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48924a41afc743139fc284a0842f0ced', 'request_id': 'req-14b514b3-c885-4267-8dda-558bc4a1a0cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:56.994817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:56.995580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:56.995938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:56.996291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] PATH_INFO: `/v3/users/d411a7966fec4d0e86d69999f816c13d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:56.998212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d411a7966fec4d0e86d69999f816c13d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:57.000264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:57.036577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48924a41afc743139fc284a0842f0ced', 'request_id': 'req-14b514b3-c885-4267-8dda-558bc4a1a0cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.045083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48924a41afc743139fc284a0842f0ced', 'request_id': 'req-14b514b3-c885-4267-8dda-558bc4a1a0cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.058900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] The token cache is being invalidate because project 48924a41afc743139fc284a0842f0ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:57.058900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.059351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.061266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 882/1781] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:56 2022] DELETE /identity/v3/projects/48924a41afc743139fc284a0842f0ced => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:57.065181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:57.066746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:57.080746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b514b3-c885-4267-8dda-558bc4a1a0cd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:57.085997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:57.086714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:57.098973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:57.098973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] PATH_INFO: `/v3/users/cef51e846d5e41d3af6ba63519d07392` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:57.098973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef51e846d5e41d3af6ba63519d07392)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:57.098973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:57.154460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd411a7966fec4d0e86d69999f816c13d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.155128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd411a7966fec4d0e86d69999f816c13d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.162485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd411a7966fec4d0e86d69999f816c13d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.176040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd411a7966fec4d0e86d69999f816c13d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.276380 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-531929d8-ba08-423b-b152-9ae40c6e3e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd411a7966fec4d0e86d69999f816c13d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.278935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef51e846d5e41d3af6ba63519d07392'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.283960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 900/1782] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:56 2022] DELETE /identity/v3/users/d411a7966fec4d0e86d69999f816c13d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:57.286463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef51e846d5e41d3af6ba63519d07392'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.287266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef51e846d5e41d3af6ba63519d07392'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.288494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:57.288831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:57.289145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:57.308421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:57.310205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:57.336175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:57.371588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 901/1783] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:57.379555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef51e846d5e41d3af6ba63519d07392'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.380245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef51e846d5e41d3af6ba63519d07392'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:57.381515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:57.383019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:57.396304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:57.400739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 883/1784] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:57 2022] DELETE /identity/v3/users/cef51e846d5e41d3af6ba63519d07392 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:35:57.443763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e86f0-490a-41f2-b4f8-4207f8f1f2d7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:57.500351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d543e-70ac-4bbc-bd6a-4cd501ea57cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:57.501114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d543e-70ac-4bbc-bd6a-4cd501ea57cb service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:57.501468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d543e-70ac-4bbc-bd6a-4cd501ea57cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:57.501811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d543e-70ac-4bbc-bd6a-4cd501ea57cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:57.571849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d543e-70ac-4bbc-bd6a-4cd501ea57cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:57.576965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d543e-70ac-4bbc-bd6a-4cd501ea57cb service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:57.588068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 902/1785] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:35:57.900746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:57.909159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:57.915913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aded68-e0f2-4f9e-9348-a99b6a8e4725 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:57.922272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:57.922943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:57.923391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:57.923824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] PATH_INFO: `/v3/projects/4a732d61db0a44bea2406e49c4226397` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:57.925098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a732d61db0a44bea2406e49c4226397)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:57.927222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.065249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a732d61db0a44bea2406e49c4226397', 'request_id': 'req-46bacf73-4a08-4c1b-804b-72efe420525c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.133737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a732d61db0a44bea2406e49c4226397', 'request_id': 'req-46bacf73-4a08-4c1b-804b-72efe420525c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.143031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a732d61db0a44bea2406e49c4226397', 'request_id': 'req-46bacf73-4a08-4c1b-804b-72efe420525c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.164961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] The token cache is being invalidate because project 4a732d61db0a44bea2406e49c4226397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:58.165254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.166124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.169994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 884/1786] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:57 2022] DELETE /identity/v3/projects/4a732d61db0a44bea2406e49c4226397 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:58.179873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7157da1-f7d4-4331-8658-ffb1ab074530 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.186961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7157da1-f7d4-4331-8658-ffb1ab074530 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.187256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7157da1-f7d4-4331-8658-ffb1ab074530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.213538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b7157da1-f7d4-4331-8658-ffb1ab074530 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:58.214762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b7157da1-f7d4-4331-8658-ffb1ab074530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:58.244944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.246525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:58.282603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:58.360577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 903/1787] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:58.367406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacf73-4a08-4c1b-804b-72efe420525c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.380794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f4c32-af56-4acd-8daf-5efd76cc4d03 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.381091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f4c32-af56-4acd-8daf-5efd76cc4d03 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.381376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f4c32-af56-4acd-8daf-5efd76cc4d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.382244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.382994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.383338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.383690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] PATH_INFO: `/v3/projects/db52330537c74e4bb90ec6a055d97b7f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.384971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db52330537c74e4bb90ec6a055d97b7f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.387062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.403161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-079f4c32-af56-4acd-8daf-5efd76cc4d03 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:58.405603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-079f4c32-af56-4acd-8daf-5efd76cc4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:58.416046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 904/1788] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:58.422056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f4c32-af56-4acd-8daf-5efd76cc4d03 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.431712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f4c32-af56-4acd-8daf-5efd76cc4d03 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.436670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.437282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.437570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.437872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.438592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:35:58.438902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.440526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.467025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 905/1789] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:35:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:35:58.471616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.481794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10317c3a-d234-40fa-85e9-50a29371a42b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.507384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db52330537c74e4bb90ec6a055d97b7f', 'request_id': 'req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.546003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.546712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.547000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.547281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.548012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.549973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.566644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db52330537c74e4bb90ec6a055d97b7f', 'request_id': 'req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.572723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db52330537c74e4bb90ec6a055d97b7f', 'request_id': 'req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.585900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] The token cache is being invalidate because project db52330537c74e4bb90ec6a055d97b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:58.586199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.586912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.588673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 885/1790] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:58 2022] DELETE /identity/v3/projects/db52330537c74e4bb90ec6a055d97b7f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:58.593137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.594473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:58.607408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea51b-f5b9-4a26-a59c-cfe8ba004b7b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.612097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.612773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.613055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.613332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] PATH_INFO: `/v3/users/8b165af8cccd4e4884adae421fedfa61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.614516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b165af8cccd4e4884adae421fedfa61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.616432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.622566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:58.665681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 906/1791] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:35:58 2022] POST /identity/v3/projects => generated 376 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:35:58.669725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.671223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:58.747804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc91538-79e3-4651-835b-9d73d185ad62 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.752649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.753281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.753564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.753847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.754534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.756506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.854510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b165af8cccd4e4884adae421fedfa61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.866469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b165af8cccd4e4884adae421fedfa61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.867664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b165af8cccd4e4884adae421fedfa61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.868509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 907/1792] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:35:58 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:35:58.875575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.877416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:58.890497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3d02d-40ed-4c54-9cd5-9f370eb594ee admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.895598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.896429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.896738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.897027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.897740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.902597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.910454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 908/1793] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:58 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:35:58.914410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.925087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc33d7a-fc69-4ffe-8319-dd13f9e19da3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.929721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.930333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.930616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.930891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.942743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.942743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:58.947590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 909/1794] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:35:58 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:35:58.948696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:58.959566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0499a43-40fb-4008-9155-046e2ce6e485 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:58.961968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b165af8cccd4e4884adae421fedfa61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.962799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4ae77b9a-4c27-42a0-91d6-6c8543f698e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b165af8cccd4e4884adae421fedfa61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:58.976655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:58.976655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:58.976655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:58.976655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] PATH_INFO: `/v3/projects/e81be726e0c24adc8e26147ea5522e06/users/d051affe0c0c4dd19df67736f2880155/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:58.976998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:58.982953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 886/1795] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:35:58 2022] DELETE /identity/v3/users/8b165af8cccd4e4884adae421fedfa61 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:58.989343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81be726e0c24adc8e26147ea5522e06, user_id=d051affe0c0c4dd19df67736f2880155, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:58.992136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:59.070933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 910/1796] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:35:58 2022] PUT /identity/v3/projects/e81be726e0c24adc8e26147ea5522e06/users/d051affe0c0c4dd19df67736f2880155/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:59.083467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee463bb-945c-4759-a47a-438240002c3a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:59.083921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee463bb-945c-4759-a47a-438240002c3a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:59.084025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee463bb-945c-4759-a47a-438240002c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:59.113587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dee463bb-945c-4759-a47a-438240002c3a None None] MFA Rules not processed for user `d051affe0c0c4dd19df67736f2880155`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:59.115131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dee463bb-945c-4759-a47a-438240002c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:59.194589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 887/1797] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:35:59.209377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:59.211381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:59.342666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7e982-7de1-432e-b8d3-9c134ef7fac5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:59.347191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5754e6e-2847-403c-a9d7-ca5b7708a14c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:59.348013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5754e6e-2847-403c-a9d7-ca5b7708a14c service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:59.348453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5754e6e-2847-403c-a9d7-ca5b7708a14c service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:59.348783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5754e6e-2847-403c-a9d7-ca5b7708a14c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:59.360984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5754e6e-2847-403c-a9d7-ca5b7708a14c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:59.362365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5754e6e-2847-403c-a9d7-ca5b7708a14c service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:59.374247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 911/1798] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:35:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:35:59.543781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee463bb-945c-4759-a47a-438240002c3a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:35:59.545544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dee463bb-945c-4759-a47a-438240002c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:35:59.667413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee463bb-945c-4759-a47a-438240002c3a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:35:59.676584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:35:59.677484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:59.677856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:59.678256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] PATH_INFO: `/v3/projects/820d413fc6014edc92a710f001a02aca` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:59.679665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=820d413fc6014edc92a710f001a02aca)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:35:59.682046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:35:59.717243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:35:59.821520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820d413fc6014edc92a710f001a02aca', 'request_id': 'req-bb391a0d-66ba-4f78-854c-89c9f85725c8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:59.899596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820d413fc6014edc92a710f001a02aca', 'request_id': 'req-bb391a0d-66ba-4f78-854c-89c9f85725c8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:59.914653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820d413fc6014edc92a710f001a02aca', 'request_id': 'req-bb391a0d-66ba-4f78-854c-89c9f85725c8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:59.929911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] The token cache is being invalidate because project 820d413fc6014edc92a710f001a02aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:35:59.930230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:59.930885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb391a0d-66ba-4f78-854c-89c9f85725c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:35:59.932957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 888/1799] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:35:59 2022] DELETE /identity/v3/projects/820d413fc6014edc92a710f001a02aca => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:35:59.938715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2682d7-6590-4f2e-b7b9-c9345f384470 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:35:59.939180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2682d7-6590-4f2e-b7b9-c9345f384470 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:35:59.939528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2682d7-6590-4f2e-b7b9-c9345f384470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:35:59.961720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ae2682d7-6590-4f2e-b7b9-c9345f384470 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:35:59.963028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ae2682d7-6590-4f2e-b7b9-c9345f384470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:00.054016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 912/1800] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:00.063728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13d111-42e8-4f52-99a8-a2407d70cd11 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.064043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13d111-42e8-4f52-99a8-a2407d70cd11 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.064304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13d111-42e8-4f52-99a8-a2407d70cd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.090735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6f13d111-42e8-4f52-99a8-a2407d70cd11 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:00.092172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6f13d111-42e8-4f52-99a8-a2407d70cd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:00.121287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 889/1801] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:00.126870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2682d7-6590-4f2e-b7b9-c9345f384470 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:00.138281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2682d7-6590-4f2e-b7b9-c9345f384470 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:00.142832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:00.143539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.143817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.144128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.144862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:00.145167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:00.146884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:00.163763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 913/1802] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:00.164779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13d111-42e8-4f52-99a8-a2407d70cd11 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:00.179677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13d111-42e8-4f52-99a8-a2407d70cd11 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:00.192310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:00.192960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.193802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.194208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.195128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:00.197436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:00.321523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 890/1803] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:00 2022] POST /identity/v3/projects => generated 382 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:00.326035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:00.339804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c5437-69eb-4ce3-8568-6f853680e276 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:00.475349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:00.488648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:00.489372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.489721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.490071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.490971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:00.493952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:00.610443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 914/1804] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:00 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:00.615142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:00.625719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b1777-9a98-4d23-aefe-4870a142746f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:00.630331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:00.630984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.631267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.631546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.632268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:00.633898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:00.645977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 891/1805] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:00 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:00.650209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:00.661597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b7e8f-f47d-47a2-a46d-31200bad0af0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:00.666198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d3f9e-7599-46fe-b2b4-b564bb3f0ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:00.666823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d3f9e-7599-46fe-b2b4-b564bb3f0ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.673691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d3f9e-7599-46fe-b2b4-b564bb3f0ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.673691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d3f9e-7599-46fe-b2b4-b564bb3f0ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.673691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d3f9e-7599-46fe-b2b4-b564bb3f0ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:00.673691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d3f9e-7599-46fe-b2b4-b564bb3f0ec3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:00.684979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 915/1806] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:00 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:00.687279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:00.698862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cfc2d-287d-4ea2-9c13-6bcafd5d5195 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:00.704778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:00.705603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.705801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.706075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] PATH_INFO: `/v3/projects/25b7c9adef214016b292e9e531b5226a/users/5e84474c778a43ccba9599dc791ef26e/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.723706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b7c9adef214016b292e9e531b5226a, user_id=5e84474c778a43ccba9599dc791ef26e, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:00.726574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:00.800551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 892/1807] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:00 2022] PUT /identity/v3/projects/25b7c9adef214016b292e9e531b5226a/users/5e84474c778a43ccba9599dc791ef26e/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:00.812651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cd988-eae1-458c-82d1-44991186d8d8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:00.813563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cd988-eae1-458c-82d1-44991186d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:00.814076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cd988-eae1-458c-82d1-44991186d8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:00.863016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-358cd988-eae1-458c-82d1-44991186d8d8 None None] MFA Rules not processed for user `5e84474c778a43ccba9599dc791ef26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:00.864219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-358cd988-eae1-458c-82d1-44991186d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:00.960544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 916/1808] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:00.961449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:01.033507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a826719-1bd7-42a9-89f6-f01625316cd4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:01.038472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:01.039076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:01.039341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:01.039621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:01.040423 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:01.042503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:01.129014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 893/1809] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:00 2022] POST /identity/v3/projects => generated 380 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:01.134304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cd988-eae1-458c-82d1-44991186d8d8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:01.147631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cd988-eae1-458c-82d1-44991186d8d8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:01.218723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:01.219179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:01.219686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:01.220077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:01.221106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:01.223139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:01.401142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 917/1810] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:01 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:01.402377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:01.423290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68077-4014-454f-8267-90393045e9be admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:01.434078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9b010-8c5d-4579-9737-553116fb5382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:01.435856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9b010-8c5d-4579-9737-553116fb5382 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:01.436481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9b010-8c5d-4579-9737-553116fb5382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:01.437365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9b010-8c5d-4579-9737-553116fb5382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:01.438499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9b010-8c5d-4579-9737-553116fb5382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:01.445433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9b010-8c5d-4579-9737-553116fb5382 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:01.499439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 894/1811] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:01 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:01.499994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:01.504555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363222b-e1ba-4817-bb26-454bebcaaa84 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:01.512988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:01.514195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:01.515353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:01.515753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] PATH_INFO: `/v3/projects/20d239aa859b45eaab9236b8f50c8e5c/users/fba5b5d319ee48e7b4eca4f053428f4c/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:01.585431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d239aa859b45eaab9236b8f50c8e5c, user_id=fba5b5d319ee48e7b4eca4f053428f4c, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:01.588442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:01.624502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c4a3e-ee06-45a6-b050-2cd9cc8c5fdc None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:01.624819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c4a3e-ee06-45a6-b050-2cd9cc8c5fdc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:01.625107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c4a3e-ee06-45a6-b050-2cd9cc8c5fdc None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:01.626900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 895/1812] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:36:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:36:01.636857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:01.637622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:01.638075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:01.662724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:01.664173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:01.677412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:01.775140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 896/1813] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:36:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:01.788923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:01.790577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:01.806215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb1d4b-dcc5-4fbe-ab6d-a51a7bdb1b56 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:01.813477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:01.814188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:01.814487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:01.814765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:01.831120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:01.875505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:01.875505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 897/1814] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:02.344531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 918/1815] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:01 2022] PUT /identity/v3/projects/20d239aa859b45eaab9236b8f50c8e5c/users/fba5b5d319ee48e7b4eca4f053428f4c/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:02.347641 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:02.360143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7af1a-edf4-42a8-988d-a2fc0b46e878 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:02.435346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc69e64-68bb-4d42-8b56-08c30b426dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:02.435346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc69e64-68bb-4d42-8b56-08c30b426dca admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:02.435346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc69e64-68bb-4d42-8b56-08c30b426dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:02.435346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc69e64-68bb-4d42-8b56-08c30b426dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:02.436184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc69e64-68bb-4d42-8b56-08c30b426dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:02.438371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc69e64-68bb-4d42-8b56-08c30b426dca admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:02.452255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 898/1816] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:02 2022] GET /identity/v3/roles => generated 1427 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:02.460997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:02.472385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff718e-0003-4477-a1ff-9c7c251579f6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:02.478072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:02.479223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:02.480121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:02.481157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] PATH_INFO: `/v3/projects/20d239aa859b45eaab9236b8f50c8e5c/users/fba5b5d319ee48e7b4eca4f053428f4c/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:02.487855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d239aa859b45eaab9236b8f50c8e5c, user_id=fba5b5d319ee48e7b4eca4f053428f4c, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:02.492809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:02.505911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 919/1817] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:02 2022] PUT /identity/v3/projects/20d239aa859b45eaab9236b8f50c8e5c/users/fba5b5d319ee48e7b4eca4f053428f4c/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:02.520339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a061b-f59b-4262-be73-d32ffa6ef23b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:02.520339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a061b-f59b-4262-be73-d32ffa6ef23b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:02.520339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a061b-f59b-4262-be73-d32ffa6ef23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:02.560879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b29a061b-f59b-4262-be73-d32ffa6ef23b None None] MFA Rules not processed for user `fba5b5d319ee48e7b4eca4f053428f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:02.562163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b29a061b-f59b-4262-be73-d32ffa6ef23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:02.664532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 899/1818] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:02.684520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:02.686462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:02.708788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:02.761928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4640086-2ecd-4dd7-a2ae-fcfaa7037881 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:02.767091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:02.767897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:02.768376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:02.768845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:02.813608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:02.813608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:02.829426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 920/1819] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:03.942020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a061b-f59b-4262-be73-d32ffa6ef23b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:03.971312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a061b-f59b-4262-be73-d32ffa6ef23b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:04.000105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba5b5d319ee48e7b4eca4f053428f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20d239aa859b45eaab9236b8f50c8e5c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:04.003746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:04.004203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:04.004771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:04.005754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:04.006294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:04.008378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:04.026732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 900/1820] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:04.032718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:04.045433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12d1a6-205c-46ff-b737-eaf342c6b70d service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:04.052178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba5b5d319ee48e7b4eca4f053428f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20d239aa859b45eaab9236b8f50c8e5c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:04.052990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:04.053526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:04.054032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:04.054965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:04.070194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:04.249149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 921/1821] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:04 2022] POST /identity/v3/projects => generated 353 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:04.250538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:04.257496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:04.273497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bee49-76df-493d-8d0b-7788fcbfbaa5 tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:04.354518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba5b5d319ee48e7b4eca4f053428f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20d239aa859b45eaab9236b8f50c8e5c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:04.355239 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:04.355593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:04.356030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:04.357001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:04.357403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:04.359215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:04.386422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 901/1822] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:04.442796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:04.477914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034465-b417-425a-9070-dc87e21921fe tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:04.485381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba5b5d319ee48e7b4eca4f053428f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20d239aa859b45eaab9236b8f50c8e5c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:04.486698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:04.486698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:04.486904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] PATH_INFO: `/v3/projects/9c6c572c83c543ec8bff30aefb36ecee` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:04.488116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9c6c572c83c543ec8bff30aefb36ecee)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:04.490259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:04.602757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6c572c83c543ec8bff30aefb36ecee', 'request_id': 'req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:04.615772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6c572c83c543ec8bff30aefb36ecee', 'request_id': 'req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:04.623673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6c572c83c543ec8bff30aefb36ecee', 'request_id': 'req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:04.707388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] The token cache is being invalidate because project 9c6c572c83c543ec8bff30aefb36ecee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:04.707929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:04.709402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-18eac6c5-7d8d-480d-bf39-3ffff4ba577e tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:04.711229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 922/1823] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:04 2022] DELETE /identity/v3/projects/9c6c572c83c543ec8bff30aefb36ecee => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:05.427865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:05.429854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:05.521296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54656b5-b09c-462f-8ddc-3bd75716096d tempest-VolumeQuotaClassesTest-558874706 tempest-VolumeQuotaClassesTest-558874706-project-admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:05.597876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:05.598542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:05.598836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:05.599112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] PATH_INFO: `/v3/users/5e84474c778a43ccba9599dc791ef26e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:05.600371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e84474c778a43ccba9599dc791ef26e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:05.602325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:05.812788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:05.816019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e84474c778a43ccba9599dc791ef26e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:05.835808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e84474c778a43ccba9599dc791ef26e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:05.836937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e84474c778a43ccba9599dc791ef26e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:05.928734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e84474c778a43ccba9599dc791ef26e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:05.929694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e84474c778a43ccba9599dc791ef26e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:05.950252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 902/1824] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:05 2022] DELETE /identity/v3/users/5e84474c778a43ccba9599dc791ef26e => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:05.955306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:05.956281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:05.956853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:05.994967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:05.996243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:06.079635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 923/1825] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:06.090830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:06.092596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:06.170068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b6481-58c6-4480-9df3-3eca24f6dbb6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:06.174796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:06.175645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:06.176043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:06.176514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:06.190412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:06.193456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:06.205946 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 903/1826] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:06.797138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:06.798812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:06.825422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68c602-e8c3-410c-b159-dd7c3f88fc2b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:06.831602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:06.832341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:06.832742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:06.833445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] PATH_INFO: `/v3/projects/25b7c9adef214016b292e9e531b5226a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:06.834868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b7c9adef214016b292e9e531b5226a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:06.836926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:06.864846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:07.013242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b7c9adef214016b292e9e531b5226a', 'request_id': 'req-021e827d-7151-4a79-a24c-b92ecd5424b6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.022148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b7c9adef214016b292e9e531b5226a', 'request_id': 'req-021e827d-7151-4a79-a24c-b92ecd5424b6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.030934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b7c9adef214016b292e9e531b5226a', 'request_id': 'req-021e827d-7151-4a79-a24c-b92ecd5424b6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.129701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] The token cache is being invalidate because project 25b7c9adef214016b292e9e531b5226a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:07.130066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.131370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.135484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 924/1827] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:06 2022] DELETE /identity/v3/projects/25b7c9adef214016b292e9e531b5226a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:07.139455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:07.156924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a43176-acb4-44a9-bc3e-c077f53b134a service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:07.317280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:07.318051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:07.318344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:07.318855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] PATH_INFO: `/v3/users/fba5b5d319ee48e7b4eca4f053428f4c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:07.320173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba5b5d319ee48e7b4eca4f053428f4c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:07.322253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:07.571241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba5b5d319ee48e7b4eca4f053428f4c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.584381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba5b5d319ee48e7b4eca4f053428f4c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.585229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba5b5d319ee48e7b4eca4f053428f4c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.663611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba5b5d319ee48e7b4eca4f053428f4c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.664421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d53be242-652f-466d-a52b-3aef818d7436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba5b5d319ee48e7b4eca4f053428f4c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:07.671250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:07.689858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 904/1828] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:07 2022] DELETE /identity/v3/users/fba5b5d319ee48e7b4eca4f053428f4c => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:08.357975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:08.360359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:08.403998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e827d-7151-4a79-a24c-b92ecd5424b6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:08.523824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:08.524517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:08.524905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:08.525343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] PATH_INFO: `/v3/projects/20d239aa859b45eaab9236b8f50c8e5c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:08.530288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20d239aa859b45eaab9236b8f50c8e5c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:08.531205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:08.691486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20d239aa859b45eaab9236b8f50c8e5c', 'request_id': 'req-b3f67f36-584a-4029-ace5-3a645efa5f89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:08.699066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20d239aa859b45eaab9236b8f50c8e5c', 'request_id': 'req-b3f67f36-584a-4029-ace5-3a645efa5f89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:08.706171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20d239aa859b45eaab9236b8f50c8e5c', 'request_id': 'req-b3f67f36-584a-4029-ace5-3a645efa5f89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:08.796611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] The token cache is being invalidate because project 20d239aa859b45eaab9236b8f50c8e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:08.796905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:08.797569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b3f67f36-584a-4029-ace5-3a645efa5f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:08.799084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 925/1829] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:08 2022] DELETE /identity/v3/projects/20d239aa859b45eaab9236b8f50c8e5c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:08.804163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c81443-cf6a-4ea1-b410-6dc7fbdb85ad None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:08.804653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c81443-cf6a-4ea1-b410-6dc7fbdb85ad None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:08.804760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c81443-cf6a-4ea1-b410-6dc7fbdb85ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:08.826245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-06c81443-cf6a-4ea1-b410-6dc7fbdb85ad None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:08.827511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-06c81443-cf6a-4ea1-b410-6dc7fbdb85ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:08.903770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 905/1830] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:08.912895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06a2eb-48cc-4dd8-8474-6eeda9bacbee None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:08.915231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06a2eb-48cc-4dd8-8474-6eeda9bacbee None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:08.915765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06a2eb-48cc-4dd8-8474-6eeda9bacbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:08.948798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ca06a2eb-48cc-4dd8-8474-6eeda9bacbee None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:08.950725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ca06a2eb-48cc-4dd8-8474-6eeda9bacbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:08.963759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 926/1831] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:08.969696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c81443-cf6a-4ea1-b410-6dc7fbdb85ad None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:08.985355 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c81443-cf6a-4ea1-b410-6dc7fbdb85ad None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:08.990844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:08.991504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:08.991783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:08.992055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:08.992831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:08.993123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:08.994730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:09.008537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 906/1832] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:09.016047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06a2eb-48cc-4dd8-8474-6eeda9bacbee None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:09.029750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06a2eb-48cc-4dd8-8474-6eeda9bacbee None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:09.038377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:09.039364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:09.039851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:09.040302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:09.042133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:09.048428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:09.161329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 927/1833] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:09 2022] POST /identity/v3/projects => generated 386 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:09.165084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:09.186308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21817d-83c4-4a3e-a0fa-3bbf1f41a63b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:09.299781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:09.300544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:09.326846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:09.326846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:09.327076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:09.329112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:09.441169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:09.441704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 907/1834] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:09 2022] POST /identity/v3/users => generated 294 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:09.454057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5945f02-fb2a-48f1-bf18-b161704b9f9c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:09.457497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:09.461093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:09.462053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:09.462576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:09.463068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:09.464012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:09.467139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:09.486661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 928/1835] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:09 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:09.488551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:09.506324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a61c69-823f-45ff-a5e6-6f73a073eb15 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:09.522232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142844b-42b9-4add-997d-85d852c367d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:09.523366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142844b-42b9-4add-997d-85d852c367d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:09.524027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142844b-42b9-4add-997d-85d852c367d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:09.524626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142844b-42b9-4add-997d-85d852c367d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:09.529255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142844b-42b9-4add-997d-85d852c367d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:09.531791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142844b-42b9-4add-997d-85d852c367d0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:09.578249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 908/1836] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:09 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:09.585125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:09.595834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54b34b-3e82-404a-a2d0-dff7f2953f78 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:09.612980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:09.614297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:09.614692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:09.615081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] PATH_INFO: `/v3/projects/c29006635a1c499d991105f5c713ef50/users/83aaf1ae77d04f8499b0b83fa0a8d828/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:09.645436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29006635a1c499d991105f5c713ef50, user_id=83aaf1ae77d04f8499b0b83fa0a8d828, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:09.648498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:09.797420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55da5-3804-43a1-994c-97eca077cd26 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:09.797910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55da5-3804-43a1-994c-97eca077cd26 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:09.798184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55da5-3804-43a1-994c-97eca077cd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:09.801063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 929/1837] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:09 2022] PUT /identity/v3/projects/c29006635a1c499d991105f5c713ef50/users/83aaf1ae77d04f8499b0b83fa0a8d828/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:09.833848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-60e55da5-3804-43a1-994c-97eca077cd26 None None] MFA Rules not processed for user `83aaf1ae77d04f8499b0b83fa0a8d828`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:09.835222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-60e55da5-3804-43a1-994c-97eca077cd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:09.893920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:09.939305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 909/1838] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:09.940780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:10.028148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d125406-91d9-459c-8f3b-da76179070c2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:10.028148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:10.028148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:10.028148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:10.030253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:10.030253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:10.031367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:10.141723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 930/1839] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:09 2022] POST /identity/v3/projects => generated 388 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:10.146060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e55da5-3804-43a1-994c-97eca077cd26 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:10.158405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e55da5-3804-43a1-994c-97eca077cd26 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:10.222342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:10.222977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:10.223266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:10.223548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:10.224234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:10.226293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:10.330845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 910/1840] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:10 2022] POST /identity/v3/users => generated 295 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:10.339188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:10.366415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4114-1f61-444d-95d2-643d83eb91a3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:10.381472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:10.382297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:10.383085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:10.384506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:10.385438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:10.392136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:10.416698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 931/1841] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:10 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:10.431058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:10.445572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af0e6-cbff-429a-86fe-4aee6e74c6e8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:10.463958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec048b-11e1-47ee-b1c4-0aae822a248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:10.464711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec048b-11e1-47ee-b1c4-0aae822a248c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:10.465661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec048b-11e1-47ee-b1c4-0aae822a248c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:10.466283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec048b-11e1-47ee-b1c4-0aae822a248c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:10.467117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec048b-11e1-47ee-b1c4-0aae822a248c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:10.470056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec048b-11e1-47ee-b1c4-0aae822a248c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:10.489624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 911/1842] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:10 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:10.497156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:10.523092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad2e7b-0297-43da-bfe7-d2b8e79be7f6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:10.529730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:10.530597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:10.530745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:10.530975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] PATH_INFO: `/v3/projects/46e06676017a4f71896e0ef205677ea2/users/eff0d996bae541468049487a617face5/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:10.548108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e06676017a4f71896e0ef205677ea2, user_id=eff0d996bae541468049487a617face5, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:10.551240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:10.625069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 932/1843] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:10 2022] PUT /identity/v3/projects/46e06676017a4f71896e0ef205677ea2/users/eff0d996bae541468049487a617face5/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:10.639433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dad0c-b41e-4b85-948c-0e98e5c9f53e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:10.639726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dad0c-b41e-4b85-948c-0e98e5c9f53e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:10.640002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dad0c-b41e-4b85-948c-0e98e5c9f53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:10.670444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-415dad0c-b41e-4b85-948c-0e98e5c9f53e None None] MFA Rules not processed for user `eff0d996bae541468049487a617face5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:10.671674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-415dad0c-b41e-4b85-948c-0e98e5c9f53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:10.745543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 912/1844] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:10.752891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:10.838954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0710-eb01-4a59-b187-26c397e7e04c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:10.845832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:10.846588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:10.846882 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:10.847170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:10.847993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:10.850768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:10.935097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 933/1845] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:10 2022] POST /identity/v3/projects => generated 388 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:10.939898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dad0c-b41e-4b85-948c-0e98e5c9f53e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:10.954195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dad0c-b41e-4b85-948c-0e98e5c9f53e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:11.006715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:11.020805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:11.021500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:11.021791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:11.022071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:11.022764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:11.024885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:11.135018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 913/1846] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:10 2022] POST /identity/v3/users => generated 301 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:11.144210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:11.159387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:11.163368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2269d9a-7b3f-48b1-992d-04ad2d28b7d9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:11.168894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:11.169547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:11.169903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:11.170242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:11.171048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:11.172733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:11.190371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 934/1847] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:11 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:11.191665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:11.203364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb90d4-8f37-49bc-be3c-e3c9de99eec4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:11.208125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:11.208842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:11.209130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:11.209410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] PATH_INFO: `/v3/projects/3007242357af4159b0bf8be2d47cb58c/users/a352eca25d2a45e0b1ac240087bbab78/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:11.225451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3007242357af4159b0bf8be2d47cb58c, user_id=a352eca25d2a45e0b1ac240087bbab78, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:11.228113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:11.304508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 914/1848] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:11 2022] PUT /identity/v3/projects/3007242357af4159b0bf8be2d47cb58c/users/a352eca25d2a45e0b1ac240087bbab78/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:11.305575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:11.317668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a5892-0d7b-4997-b06f-c4e1945a6184 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:11.387063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ee21b-b0d6-4a00-8317-29c97103422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:11.387697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ee21b-b0d6-4a00-8317-29c97103422c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:11.387986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ee21b-b0d6-4a00-8317-29c97103422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:11.388448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ee21b-b0d6-4a00-8317-29c97103422c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:11.389188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ee21b-b0d6-4a00-8317-29c97103422c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:11.390808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ee21b-b0d6-4a00-8317-29c97103422c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:11.401112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 935/1849] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:11 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:11.407779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:11.419164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7a719-5093-414a-9e20-2dbadcf77caf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:11.423845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:11.424549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:11.425003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:11.425288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] PATH_INFO: `/v3/projects/3007242357af4159b0bf8be2d47cb58c/users/a352eca25d2a45e0b1ac240087bbab78/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:11.427255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3007242357af4159b0bf8be2d47cb58c, user_id=a352eca25d2a45e0b1ac240087bbab78, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:11.429950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:11.442137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 915/1850] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:11 2022] PUT /identity/v3/projects/3007242357af4159b0bf8be2d47cb58c/users/a352eca25d2a45e0b1ac240087bbab78/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:11.453084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:11.453374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:11.453662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:11.487333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] MFA Rules not processed for user `a352eca25d2a45e0b1ac240087bbab78`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:11.488636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:11.575327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 936/1851] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:11.590472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:11.593087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:11.694981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f201c-2e38-48a7-bfbf-ea133ea3b2a9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:11.698740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:11.700544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:11.701293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:11.701577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:11.716249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:11.717967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:11.731824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 916/1852] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:13.039234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:13.045824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:13.060809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957bcfb-e71c-4247-bf82-fd10524cda26 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:13.066291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:13.067352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:13.067745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:13.068220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:13.151578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:13.155215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:13.169395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 937/1853] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:18.880230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:18.889775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ec99b-d0b6-4d4a-8a8b-88b80cc32714 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:18.893438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff34d1-90e5-493f-bead-dac4853ac769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:18.894062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff34d1-90e5-493f-bead-dac4853ac769 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:18.894343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff34d1-90e5-493f-bead-dac4853ac769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:18.894615 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff34d1-90e5-493f-bead-dac4853ac769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:18.958156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff34d1-90e5-493f-bead-dac4853ac769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:18.959659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff34d1-90e5-493f-bead-dac4853ac769 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:18.971602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 917/1854] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:23.163106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:28.964725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:32.172739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:32.241982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68002c-f439-4ea3-bfc9-36939d4d5353 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:32.246589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:32.247238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:32.247518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:32.247790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] PATH_INFO: `/v3/users/83aaf1ae77d04f8499b0b83fa0a8d828` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:32.249108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83aaf1ae77d04f8499b0b83fa0a8d828)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:32.250966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:32.471981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83aaf1ae77d04f8499b0b83fa0a8d828'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:32.473149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83aaf1ae77d04f8499b0b83fa0a8d828'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:32.482842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83aaf1ae77d04f8499b0b83fa0a8d828'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:32.498800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83aaf1ae77d04f8499b0b83fa0a8d828'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:32.612566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83aaf1ae77d04f8499b0b83fa0a8d828'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:32.620297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 938/1855] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:32 2022] DELETE /identity/v3/users/83aaf1ae77d04f8499b0b83fa0a8d828 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:32.625518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e466de5-9c32-4275-8964-699607605d3d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:32.626077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e466de5-9c32-4275-8964-699607605d3d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:32.626077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e466de5-9c32-4275-8964-699607605d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:32.648238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7e466de5-9c32-4275-8964-699607605d3d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:32.649459 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7e466de5-9c32-4275-8964-699607605d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:32.714485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 918/1856] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:32.725074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:32.726536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:32.802137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5db83c-b204-4a49-bb56-583acf286323 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:32.802137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c18d29-e7e9-4665-a69f-35f76c1c709c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:32.802137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c18d29-e7e9-4665-a69f-35f76c1c709c service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:32.802448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c18d29-e7e9-4665-a69f-35f76c1c709c service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:32.802694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c18d29-e7e9-4665-a69f-35f76c1c709c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:32.813306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c18d29-e7e9-4665-a69f-35f76c1c709c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:32.814777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c18d29-e7e9-4665-a69f-35f76c1c709c service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:32.825157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 939/1857] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:32.851130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e466de5-9c32-4275-8964-699607605d3d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:32.852816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7e466de5-9c32-4275-8964-699607605d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:32.865870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e466de5-9c32-4275-8964-699607605d3d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:32.871315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:32.873126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:32.873835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:32.874118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] PATH_INFO: `/v3/users/d051affe0c0c4dd19df67736f2880155` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:32.875372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d051affe0c0c4dd19df67736f2880155)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:32.877353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:33.130935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd051affe0c0c4dd19df67736f2880155'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.141466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd051affe0c0c4dd19df67736f2880155'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.142273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd051affe0c0c4dd19df67736f2880155'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.245627 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd051affe0c0c4dd19df67736f2880155'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.246449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd051affe0c0c4dd19df67736f2880155'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.275128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 919/1858] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:32 2022] DELETE /identity/v3/users/d051affe0c0c4dd19df67736f2880155 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:33.281890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57667424-42e3-4972-b192-cc4739331e17 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:33.282437 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57667424-42e3-4972-b192-cc4739331e17 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:33.282548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57667424-42e3-4972-b192-cc4739331e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:33.305360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-57667424-42e3-4972-b192-cc4739331e17 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:33.306727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-57667424-42e3-4972-b192-cc4739331e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:33.327534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:33.364007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:33.365482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:33.380677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 940/1859] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:33.395217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57667424-42e3-4972-b192-cc4739331e17 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:33.397994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-57667424-42e3-4972-b192-cc4739331e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:33.451462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900454a-ef8a-446c-902f-869dbb8f00b4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:33.458718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:33.459409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:33.459938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:33.460368 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] PATH_INFO: `/v3/projects/c29006635a1c499d991105f5c713ef50` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:33.466945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29006635a1c499d991105f5c713ef50)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:33.466945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:33.485340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:33.486095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57667424-42e3-4972-b192-cc4739331e17 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:33.490074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:33.490836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:33.491501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:33.491501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:33.502832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:33.504328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:33.515049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 941/1860] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:33.639992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29006635a1c499d991105f5c713ef50', 'request_id': 'req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.719687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29006635a1c499d991105f5c713ef50', 'request_id': 'req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.729326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29006635a1c499d991105f5c713ef50', 'request_id': 'req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.746305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] The token cache is being invalidate because project c29006635a1c499d991105f5c713ef50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:33.746811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.748429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:33.750390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 920/1861] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:33 2022] DELETE /identity/v3/projects/c29006635a1c499d991105f5c713ef50 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:33.754905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:33.766544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f974924-40db-4f0a-9023-b7d7750b1aff service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:33.862962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:33.863702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:33.863900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:33.864174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] PATH_INFO: `/v3/users/eff0d996bae541468049487a617face5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:33.865723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff0d996bae541468049487a617face5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:33.867523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:33.986436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:33.987952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:34.002588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067f1d2-df2e-4e2f-aed6-5a9778fdf8c0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:34.007506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:34.008171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.008552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.008770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] PATH_INFO: `/v3/projects/e81be726e0c24adc8e26147ea5522e06` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.009987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81be726e0c24adc8e26147ea5522e06)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:34.011995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:34.138830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff0d996bae541468049487a617face5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.139887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff0d996bae541468049487a617face5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.140644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81be726e0c24adc8e26147ea5522e06', 'request_id': 'req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.150919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff0d996bae541468049487a617face5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.159226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:34.164381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff0d996bae541468049487a617face5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.194696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81be726e0c24adc8e26147ea5522e06', 'request_id': 'req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.202630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81be726e0c24adc8e26147ea5522e06', 'request_id': 'req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.222040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] The token cache is being invalidate because project e81be726e0c24adc8e26147ea5522e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:34.222040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.222040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f3b2956-4b84-4d3f-b807-b56c9a49f0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.222040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 921/1862] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:33 2022] DELETE /identity/v3/projects/e81be726e0c24adc8e26147ea5522e06 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:34.223258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe343d9-d767-4610-a9e4-6ff6d916dd6e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.223368 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe343d9-d767-4610-a9e4-6ff6d916dd6e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.223649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe343d9-d767-4610-a9e4-6ff6d916dd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.244448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dfe343d9-d767-4610-a9e4-6ff6d916dd6e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:34.245713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dfe343d9-d767-4610-a9e4-6ff6d916dd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:34.296812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6dc85b22-b945-434d-bd3e-1ad0923e5002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff0d996bae541468049487a617face5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:34.307592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 942/1863] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:33 2022] DELETE /identity/v3/users/eff0d996bae541468049487a617face5 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:34.322594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 922/1864] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:34.326246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc14ef7-9b1a-4b7c-9bc8-4648310d7848 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.328066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc14ef7-9b1a-4b7c-9bc8-4648310d7848 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.328557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc14ef7-9b1a-4b7c-9bc8-4648310d7848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.363579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6cc14ef7-9b1a-4b7c-9bc8-4648310d7848 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:34.366031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6cc14ef7-9b1a-4b7c-9bc8-4648310d7848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:34.432280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 943/1865] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:34.437102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe343d9-d767-4610-a9e4-6ff6d916dd6e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:34.446949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe343d9-d767-4610-a9e4-6ff6d916dd6e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:34.452724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:34.453349 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.453631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.453906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.454660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:34.454967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:34.456611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:34.473463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 923/1866] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:34.474632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc14ef7-9b1a-4b7c-9bc8-4648310d7848 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:34.484495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc14ef7-9b1a-4b7c-9bc8-4648310d7848 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:34.489273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:34.489926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.490200 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.490477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.491209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:34.493582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:34.595957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 944/1867] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:34 2022] POST /identity/v3/projects => generated 360 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:34.600077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:34.610966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2e221-96a5-468a-a75d-ef88131597f5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:34.620739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:34.670363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:34.671022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.671223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.671486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.672226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:34.674311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:34.818160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 924/1868] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:34 2022] POST /identity/v3/users => generated 281 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:34.823338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:34.836654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c610158-0d0b-429a-8868-23014b210c4c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:34.843916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:34.844594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.844880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.845156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.846150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:34.847799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:34.864971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 945/1869] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:34 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:34.869639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:34.880374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af6e9e-0815-4999-96e1-ad105907b961 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:34.885364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:34.886179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.886353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.886642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.888215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:34.890947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:34.904609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 925/1870] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:34 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:34.909168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:34.920262 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df695fad-6c74-40aa-b761-4b11acab23bb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:34.925126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166a006-4a81-4eef-b551-e36c101b77cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:34.925782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a006-4a81-4eef-b551-e36c101b77cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:34.926071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a006-4a81-4eef-b551-e36c101b77cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:34.926354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a006-4a81-4eef-b551-e36c101b77cf admin admin] PATH_INFO: `/v3/projects/9ba75530fc1c4a41a5ceca3b9df5d873/users/74066373d8d4426a9f2b7e32e0718d63/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:34.948833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166a006-4a81-4eef-b551-e36c101b77cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba75530fc1c4a41a5ceca3b9df5d873, user_id=74066373d8d4426a9f2b7e32e0718d63, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:34.952914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166a006-4a81-4eef-b551-e36c101b77cf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:35.016569 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:35.019109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:35.030771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 946/1871] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:34 2022] PUT /identity/v3/projects/9ba75530fc1c4a41a5ceca3b9df5d873/users/74066373d8d4426a9f2b7e32e0718d63/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:35.040786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcd8e6-d5a3-440b-b37c-b204e7beed3d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:35.049089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:35.049089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:35.049089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:35.084418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] MFA Rules not processed for user `74066373d8d4426a9f2b7e32e0718d63`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:35.086260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:35.147033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:35.147811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:35.148170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:35.148557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] PATH_INFO: `/v3/projects/46e06676017a4f71896e0ef205677ea2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:35.149838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e06676017a4f71896e0ef205677ea2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:35.151856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:35.180679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 947/1872] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:35.217135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:35.218574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:35.248242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:35.265563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e06676017a4f71896e0ef205677ea2', 'request_id': 'req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.286644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e029de-3630-4f02-81e5-3069c4b4fea8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:35.336632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e06676017a4f71896e0ef205677ea2', 'request_id': 'req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.344950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e06676017a4f71896e0ef205677ea2', 'request_id': 'req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.354049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:35.356722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:35.356722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:35.356722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:35.358906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] The token cache is being invalidate because project 46e06676017a4f71896e0ef205677ea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:35.359209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.359892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.361675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 926/1873] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:35 2022] DELETE /identity/v3/projects/46e06676017a4f71896e0ef205677ea2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:35.365311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:35.366726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:35.395523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:35.419425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:35.421055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:35.430501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 948/1874] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:35.438085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b2259-b8e6-4c4e-a6e9-5b22784a7174 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:35.451534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:35.451534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:35.451534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:35.451534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] PATH_INFO: `/v3/users/a352eca25d2a45e0b1ac240087bbab78` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:35.452566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a352eca25d2a45e0b1ac240087bbab78)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:35.452566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:35.453604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:35.469861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119617a-ff5b-4d36-a922-8330cff26348 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:35.476262 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:35.477088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:35.477451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:35.477877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] PATH_INFO: `/v3/users/74066373d8d4426a9f2b7e32e0718d63` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:35.479243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=74066373d8d4426a9f2b7e32e0718d63)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:35.482011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:35.673370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74066373d8d4426a9f2b7e32e0718d63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.678502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74066373d8d4426a9f2b7e32e0718d63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.681601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74066373d8d4426a9f2b7e32e0718d63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.698474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74066373d8d4426a9f2b7e32e0718d63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.739580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a352eca25d2a45e0b1ac240087bbab78'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.753183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a352eca25d2a45e0b1ac240087bbab78'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.754833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a352eca25d2a45e0b1ac240087bbab78'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.765436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6437a90f-2e14-4543-8097-ef79e24484bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74066373d8d4426a9f2b7e32e0718d63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.767905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:35.772430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 949/1875] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:35 2022] DELETE /identity/v3/users/74066373d8d4426a9f2b7e32e0718d63 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:35.777726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:35.777889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:35.778722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:35.799732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:35.810622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:35.845546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a352eca25d2a45e0b1ac240087bbab78'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.847064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a352eca25d2a45e0b1ac240087bbab78'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:35.865020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 950/1876] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:35.866839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 927/1877] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:35 2022] DELETE /identity/v3/users/a352eca25d2a45e0b1ac240087bbab78 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:35.877517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:35.879282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:35.943637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c6ee1-a16b-4341-bb60-5f23953697b7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:36.000447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:36.001123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.001413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.001698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.088081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:36.089963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:36.101466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 951/1878] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:36.463047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:36.465265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:36.494092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452aeb7-79c7-4bd3-bc4a-5e4e7a6955e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:36.499344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:36.500008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.500354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.500680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] PATH_INFO: `/v3/projects/3007242357af4159b0bf8be2d47cb58c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.501861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3007242357af4159b0bf8be2d47cb58c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:36.503852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:36.584448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:36.585989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:36.600643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f3f18-b0c9-4849-8951-9a05b67dc8cd service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:36.621154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:36.621154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.621154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.621154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] PATH_INFO: `/v3/projects/9ba75530fc1c4a41a5ceca3b9df5d873` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.621862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ba75530fc1c4a41a5ceca3b9df5d873)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:36.621862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:36.621862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:36.648724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3007242357af4159b0bf8be2d47cb58c', 'request_id': 'req-8714b709-212b-4200-a70d-4dba48643d72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.748550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3007242357af4159b0bf8be2d47cb58c', 'request_id': 'req-8714b709-212b-4200-a70d-4dba48643d72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.754953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3007242357af4159b0bf8be2d47cb58c', 'request_id': 'req-8714b709-212b-4200-a70d-4dba48643d72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.768687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] The token cache is being invalidate because project 3007242357af4159b0bf8be2d47cb58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:36.768994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.769731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8714b709-212b-4200-a70d-4dba48643d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.771275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 928/1879] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:36 2022] DELETE /identity/v3/projects/3007242357af4159b0bf8be2d47cb58c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:36.777372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4923b-4695-47c4-8011-29b730661bfd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.777670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4923b-4695-47c4-8011-29b730661bfd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.778074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4923b-4695-47c4-8011-29b730661bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.784609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ba75530fc1c4a41a5ceca3b9df5d873', 'request_id': 'req-9bb64442-fb7e-41ae-ab81-9690823a1a60'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.792317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ba75530fc1c4a41a5ceca3b9df5d873', 'request_id': 'req-9bb64442-fb7e-41ae-ab81-9690823a1a60'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.799136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ba75530fc1c4a41a5ceca3b9df5d873', 'request_id': 'req-9bb64442-fb7e-41ae-ab81-9690823a1a60'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.799693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6cc4923b-4695-47c4-8011-29b730661bfd None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:36.800952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6cc4923b-4695-47c4-8011-29b730661bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:36.823299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:36.873090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 929/1880] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:36.883081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227265a-2779-4da2-aefc-258da4aa18ab None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.891294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227265a-2779-4da2-aefc-258da4aa18ab None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.891294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227265a-2779-4da2-aefc-258da4aa18ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.891478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] The token cache is being invalidate because project 9ba75530fc1c4a41a5ceca3b9df5d873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:36.891478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.891478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9bb64442-fb7e-41ae-ab81-9690823a1a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:36.891478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 952/1881] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:36 2022] DELETE /identity/v3/projects/9ba75530fc1c4a41a5ceca3b9df5d873 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:36.896580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a53b2-5f32-49bf-a867-458bfaf630e7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.896877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a53b2-5f32-49bf-a867-458bfaf630e7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.897348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a53b2-5f32-49bf-a867-458bfaf630e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.906327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6227265a-2779-4da2-aefc-258da4aa18ab None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:36.907565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6227265a-2779-4da2-aefc-258da4aa18ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:36.916169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 930/1882] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:36.924312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227265a-2779-4da2-aefc-258da4aa18ab None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:36.929574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-748a53b2-5f32-49bf-a867-458bfaf630e7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:36.930799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-748a53b2-5f32-49bf-a867-458bfaf630e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:36.937781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227265a-2779-4da2-aefc-258da4aa18ab None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:36.942632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:36.943257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.943550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.943822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.944780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 953/1883] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:36.944923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:36.945395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:36.947215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:36.951606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5b3da-b9fb-45d9-87db-5d8187ef1582 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.954091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5b3da-b9fb-45d9-87db-5d8187ef1582 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.954091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5b3da-b9fb-45d9-87db-5d8187ef1582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.961303 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 931/1884] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:36.966130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:36.975999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfcfe2-e3f2-4c50-9907-78208ba3a8a4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:36.976889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-27e5b3da-b9fb-45d9-87db-5d8187ef1582 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:36.978171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27e5b3da-b9fb-45d9-87db-5d8187ef1582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:36.981032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:36.982061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:36.982411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:36.982700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:36.983378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:36.985326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:36.988967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 954/1885] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:36.998781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5b3da-b9fb-45d9-87db-5d8187ef1582 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.008662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5b3da-b9fb-45d9-87db-5d8187ef1582 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.015005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.015635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.015931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.016227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.017044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:37.017359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.018968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.031199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 955/1886] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:36:37.035569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.045676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60df65-2249-4f20-b07f-2813c2bf10e6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.050425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.051027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.051354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.051601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.052269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.054228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.117938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 932/1887] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:36 2022] POST /identity/v3/projects => generated 410 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:37.117938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.182128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 956/1888] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/projects => generated 376 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:37.184732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.194576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e4afe-4184-4855-97b7-08bca1b0115b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.265668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb2b92-9f56-4ee0-a291-57e0198f484a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.270534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.271351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.271652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.272004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.272864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.274623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.279840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.284614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.284614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.284614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.284614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.284614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.389680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 957/1889] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:37.393749 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.404201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11990c8-ce67-4b8b-a9f0-02ed91eee42c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.408817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.409745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.410598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.410887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.411829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.413653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.421768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 933/1890] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/users => generated 306 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:37.424626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 958/1891] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:37 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:37.425861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.428226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.435957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464917ac-ff2d-4070-a0bb-ca064947a69a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.438347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336276e9-9c12-4c66-8d7d-0897b9f8565c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.441019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.441642 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.441941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.442221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.442956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.443338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.443931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.444215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.444524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.444714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.445357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.448233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.459180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 959/1892] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:37 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:37.461386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 934/1893] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:37 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:37.465165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.465299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.476045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7798f-6835-412e-bb35-549c5efdfe04 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.476780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c795d-f710-48ac-9d7d-1c7622757a76 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.480763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427aeff-b552-492b-ad37-46a68ea50b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.481721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427aeff-b552-492b-ad37-46a68ea50b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.481835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.482055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427aeff-b552-492b-ad37-46a68ea50b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.482334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427aeff-b552-492b-ad37-46a68ea50b0f admin admin] PATH_INFO: `/v3/projects/d92ba80d86c54035a3e58376409e9cae/users/9d3c3546f6a1448485d43619cbfb4322/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.482440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.482626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.482906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.483795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.485533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.485712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:37.497734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 935/1894] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:37 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:37.501096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427aeff-b552-492b-ad37-46a68ea50b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92ba80d86c54035a3e58376409e9cae, user_id=9d3c3546f6a1448485d43619cbfb4322, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.502146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.503803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427aeff-b552-492b-ad37-46a68ea50b0f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.511859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca90d-3044-4f1b-a141-ef3f2a26d590 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.519583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4ed15-8724-4639-9d0f-a1986d93b074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.519583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4ed15-8724-4639-9d0f-a1986d93b074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.519583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4ed15-8724-4639-9d0f-a1986d93b074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.519583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4ed15-8724-4639-9d0f-a1986d93b074 admin admin] PATH_INFO: `/v3/projects/3dbbcdfca4c24dcf875a3e28a5d4953b/users/a3fbc326e3134e02bc722bdd54ce5a32/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.538785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4ed15-8724-4639-9d0f-a1986d93b074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dbbcdfca4c24dcf875a3e28a5d4953b, user_id=a3fbc326e3134e02bc722bdd54ce5a32, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.541562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4ed15-8724-4639-9d0f-a1986d93b074 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.602096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 960/1895] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:37 2022] PUT /identity/v3/projects/d92ba80d86c54035a3e58376409e9cae/users/9d3c3546f6a1448485d43619cbfb4322/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:37.612304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432488-17da-403e-9be8-13a4316e2cf3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.612634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432488-17da-403e-9be8-13a4316e2cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.612917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432488-17da-403e-9be8-13a4316e2cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.640426 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 936/1896] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:37 2022] PUT /identity/v3/projects/3dbbcdfca4c24dcf875a3e28a5d4953b/users/a3fbc326e3134e02bc722bdd54ce5a32/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:37.649648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fd432488-17da-403e-9be8-13a4316e2cf3 None None] MFA Rules not processed for user `9d3c3546f6a1448485d43619cbfb4322`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:37.651141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fd432488-17da-403e-9be8-13a4316e2cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:37.652072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8bfd3-de68-4a3d-867b-b7e452fcbe7c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.652365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8bfd3-de68-4a3d-867b-b7e452fcbe7c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.652686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8bfd3-de68-4a3d-867b-b7e452fcbe7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.679452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:37.687562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-20f8bfd3-de68-4a3d-867b-b7e452fcbe7c None None] MFA Rules not processed for user `a3fbc326e3134e02bc722bdd54ce5a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:37.688852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-20f8bfd3-de68-4a3d-867b-b7e452fcbe7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:37.726068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 961/1897] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:37.730489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432488-17da-403e-9be8-13a4316e2cf3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.758557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 937/1898] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:37.764109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8bfd3-de68-4a3d-867b-b7e452fcbe7c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.813509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432488-17da-403e-9be8-13a4316e2cf3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.818519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.820023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.820305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.820618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.826041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.826041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.842738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8bfd3-de68-4a3d-867b-b7e452fcbe7c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:37.847725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:37.848552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:37.849097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:37.849525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:37.850560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:37.852249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:37.958760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 962/1899] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/projects => generated 370 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:37.962912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:37.992692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 938/1900] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/projects => generated 410 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:37.992692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.046294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beb37b-d1f7-4f67-b021-6fcfb35488e6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.050441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:38.053993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:38.065826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287db147-9fcf-4498-84f9-c9a9c1b93673 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.070696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.071321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.071615 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.071898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.072940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.075519 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.109133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.109773 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.110080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.110357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.112227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.114229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.224037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 939/1901] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/users => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:38.227439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.237691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d995ac-8ad8-4ca8-9ac0-a43288618095 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.242299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.242956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.243244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.243518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.244341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.246108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.256680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 940/1902] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:38 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:38.261664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.267619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 963/1903] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:37 2022] POST /identity/v3/users => generated 292 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:38.268930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.274530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d489740-27e2-43e2-a743-c6b25b4626a7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.281949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.282866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.283272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.283426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341e99-e8c9-4e20-bd98-ae773226ab4b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.283746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] PATH_INFO: `/v3/projects/3c9bb4ea9c9b48dc96aaa361de12c792/users/c691e410117f400cac136af5ac125ea2/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.290691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.291769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.291958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.292209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.293884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.300603 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.301775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9bb4ea9c9b48dc96aaa361de12c792, user_id=c691e410117f400cac136af5ac125ea2, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.305261 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.322496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 964/1904] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:38 2022] GET /identity/v3/roles => generated 1427 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:38.327862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.349016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcbb9a-5b88-484e-ba58-da6487f9af3f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.357391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.363593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.363593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.363593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] PATH_INFO: `/v3/projects/e46b3cca61ed469e92e6a629c51852fd/users/38ab66b7ee67472690f9d145268af796/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.375280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46b3cca61ed469e92e6a629c51852fd, user_id=38ab66b7ee67472690f9d145268af796, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.378573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.463754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 941/1905] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:38 2022] PUT /identity/v3/projects/3c9bb4ea9c9b48dc96aaa361de12c792/users/c691e410117f400cac136af5ac125ea2/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:38.467545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.503768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 965/1906] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:38 2022] PUT /identity/v3/projects/e46b3cca61ed469e92e6a629c51852fd/users/38ab66b7ee67472690f9d145268af796/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:38.515040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.548309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57605b8c-725b-4acd-b175-248beba2acda admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.580051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece491-7c7b-4f6e-b581-baea84eff25e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.585187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.585853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.586139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.586560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.587279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.588946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.602794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 966/1907] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:38 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:38.604256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.610713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.611351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.611639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.611920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.612725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.614338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.614574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fac4e0-87eb-4e74-b913-000f9acaeabc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.619471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c24e76-30e7-4fad-af88-b7b9ed458ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.620147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24e76-30e7-4fad-af88-b7b9ed458ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.620432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24e76-30e7-4fad-af88-b7b9ed458ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.620751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24e76-30e7-4fad-af88-b7b9ed458ed9 admin admin] PATH_INFO: `/v3/projects/e46b3cca61ed469e92e6a629c51852fd/users/38ab66b7ee67472690f9d145268af796/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.623984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c24e76-30e7-4fad-af88-b7b9ed458ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46b3cca61ed469e92e6a629c51852fd, user_id=38ab66b7ee67472690f9d145268af796, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.628696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 942/1908] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:38 2022] GET /identity/v3/roles => generated 1427 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:38.628843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c24e76-30e7-4fad-af88-b7b9ed458ed9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.629799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.638821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 967/1909] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:38 2022] PUT /identity/v3/projects/e46b3cca61ed469e92e6a629c51852fd/users/38ab66b7ee67472690f9d145268af796/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:38.641800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08548d-863b-419e-a903-84a8a4f34822 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.649971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.650429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.650777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.681740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] MFA Rules not processed for user `38ab66b7ee67472690f9d145268af796`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:38.683461 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:38.694438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:38.711250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fb682-14ee-4e69-8b77-56db13491db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.711936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fb682-14ee-4e69-8b77-56db13491db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.712227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fb682-14ee-4e69-8b77-56db13491db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.713055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fb682-14ee-4e69-8b77-56db13491db6 admin admin] PATH_INFO: `/v3/projects/3c9bb4ea9c9b48dc96aaa361de12c792/users/c691e410117f400cac136af5ac125ea2/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.715598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fb682-14ee-4e69-8b77-56db13491db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9bb4ea9c9b48dc96aaa361de12c792, user_id=c691e410117f400cac136af5ac125ea2, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.718517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fb682-14ee-4e69-8b77-56db13491db6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.725277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:38.729734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 943/1910] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:38 2022] PUT /identity/v3/projects/3c9bb4ea9c9b48dc96aaa361de12c792/users/c691e410117f400cac136af5ac125ea2/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:38.741477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.741782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.742063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.767710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 968/1911] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:38.773363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] MFA Rules not processed for user `c691e410117f400cac136af5ac125ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:38.775354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:38.778992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:38.780453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:38.838060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7726d-66e2-4bfb-9f0c-f5d80e19f7a9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:38.849515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 944/1912] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:38.890642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:38.891282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:38.891569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:38.891851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:38.949049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:38.951277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:38.960985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 969/1913] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:39.492553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:39.493460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:39.506217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999b2cf-6636-4e07-8345-808e622c8b46 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:39.510271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:39.510900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:39.511210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:39.511514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:39.522376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:39.523849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:39.544821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 945/1914] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:39 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:40.088004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:40.097417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee141f70-c1f4-4ea9-a720-5c81a402e69b service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:40.101220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:40.101844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:40.102131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:40.102410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:40.144571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:40.163335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:40.165015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:40.174650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 970/1915] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:43.985776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:43.987255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:44.058242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab0602-140e-4d50-8c98-ea30eb8ca9b1 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:44.063390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:44.064078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:44.064522 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:44.064804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] PATH_INFO: `/v3/users/858548bfa9b7455cb518849f31471865` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:44.066092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=858548bfa9b7455cb518849f31471865)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:44.070759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:44.319092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858548bfa9b7455cb518849f31471865'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:44.327232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:44.331190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858548bfa9b7455cb518849f31471865'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:44.332140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858548bfa9b7455cb518849f31471865'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:44.411070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858548bfa9b7455cb518849f31471865'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:44.412302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-38c4a4c3-d09b-4a74-83a0-fc2905e48caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858548bfa9b7455cb518849f31471865'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:44.435155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 946/1916] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:43 2022] DELETE /identity/v3/users/858548bfa9b7455cb518849f31471865 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:44.840887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:44.857220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0473b-9784-4dd8-8f72-c1f2b7bfe0c0 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:44.918092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:44.918743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:44.919557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:44.919834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] PATH_INFO: `/v3/projects/417b9d6cde90467ebe75c790867a8c89` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:44.921063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=417b9d6cde90467ebe75c790867a8c89)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:44.923115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.046212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417b9d6cde90467ebe75c790867a8c89', 'request_id': 'req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:45.052762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417b9d6cde90467ebe75c790867a8c89', 'request_id': 'req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:45.059189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417b9d6cde90467ebe75c790867a8c89', 'request_id': 'req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:45.152938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] The token cache is being invalidate because project 417b9d6cde90467ebe75c790867a8c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:45.153150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:45.154515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-07f99b9a-6d65-4c1f-be3c-1694076cdc5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:45.156346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 971/1917] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:44 2022] DELETE /identity/v3/projects/417b9d6cde90467ebe75c790867a8c89 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:45.162051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1c4b7-ff26-4168-abf7-8ea91faa0335 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.162515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1c4b7-ff26-4168-abf7-8ea91faa0335 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.162640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1c4b7-ff26-4168-abf7-8ea91faa0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.183171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6cd1c4b7-ff26-4168-abf7-8ea91faa0335 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:45.184390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6cd1c4b7-ff26-4168-abf7-8ea91faa0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:45.245526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 947/1918] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:45.253469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e777a52-9863-4a65-9ad5-db310853e859 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.253760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e777a52-9863-4a65-9ad5-db310853e859 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.254072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e777a52-9863-4a65-9ad5-db310853e859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.273925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0e777a52-9863-4a65-9ad5-db310853e859 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:45.275155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0e777a52-9863-4a65-9ad5-db310853e859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:45.284973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 972/1919] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:45.289370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1c4b7-ff26-4168-abf7-8ea91faa0335 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.299748 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1c4b7-ff26-4168-abf7-8ea91faa0335 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.304667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.305291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.305578 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.305856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.306564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:45.306873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.308503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.321912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 948/1920] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:45.327313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e777a52-9863-4a65-9ad5-db310853e859 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.336807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e777a52-9863-4a65-9ad5-db310853e859 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.343389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.344040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.344325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.344635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.345309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.347221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.417855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 973/1921] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:45 2022] POST /identity/v3/projects => generated 388 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:45.422728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.432869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5bbb-71bd-423d-86bf-a05bbf12f2af admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.492704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.493335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.493638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.493922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.494593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.496529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.587531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 949/1922] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:45 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:45.592555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.603269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b528b21-4f4d-4564-8678-80f3da52a040 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.608238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.608888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.609172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.609447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.610252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.611858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.622218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 974/1923] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:45 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:45.626214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.636776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8914a1-8986-4006-aa51-1017be8dfaff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.642594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d00eb0-0d1c-4811-a058-71d0a534b076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.643210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d00eb0-0d1c-4811-a058-71d0a534b076 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.643497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d00eb0-0d1c-4811-a058-71d0a534b076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.643776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d00eb0-0d1c-4811-a058-71d0a534b076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.644490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d00eb0-0d1c-4811-a058-71d0a534b076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.646108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d00eb0-0d1c-4811-a058-71d0a534b076 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.657726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 950/1924] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:45 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:45.661867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.672218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200517e-a75f-4f3f-9816-0a543c282a30 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.676889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.677556 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.677847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.678130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] PATH_INFO: `/v3/projects/391ac096d47d4713a72526824fcf398c/users/200b7dc83e624717a380ccddbac8ea72/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.692940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391ac096d47d4713a72526824fcf398c, user_id=200b7dc83e624717a380ccddbac8ea72, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.695704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.782833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 975/1925] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:45 2022] PUT /identity/v3/projects/391ac096d47d4713a72526824fcf398c/users/200b7dc83e624717a380ccddbac8ea72/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:45.793203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd30d3-10e2-4438-973d-56f4ba7867eb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.793485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd30d3-10e2-4438-973d-56f4ba7867eb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.800098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd30d3-10e2-4438-973d-56f4ba7867eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.811237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.813123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:45.820929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:45.837878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cecd30d3-10e2-4438-973d-56f4ba7867eb None None] MFA Rules not processed for user `200b7dc83e624717a380ccddbac8ea72`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:45.839571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cecd30d3-10e2-4438-973d-56f4ba7867eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:45.880922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:45.924298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537be8b2-3599-45aa-9abb-3d4ca6302915 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.933485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.934175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.934460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.935477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] PATH_INFO: `/v3/users/9d3c3546f6a1448485d43619cbfb4322` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.937157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d3c3546f6a1448485d43619cbfb4322)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.939093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:45.961551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 951/1926] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:45.966267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd30d3-10e2-4438-973d-56f4ba7867eb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:45.977433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd30d3-10e2-4438-973d-56f4ba7867eb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:45.982623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:45.983845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:45.984420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:45.984873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:45.985664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:45.987694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:46.114822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 952/1927] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:45 2022] POST /identity/v3/projects => generated 386 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:46.118890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:46.159092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d3c3546f6a1448485d43619cbfb4322'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:46.159931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d3c3546f6a1448485d43619cbfb4322'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:46.168969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d3c3546f6a1448485d43619cbfb4322'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:46.185863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d3c3546f6a1448485d43619cbfb4322'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:46.194173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baaf72-be65-41d6-ac41-93b52ab7be45 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:46.199434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:46.200323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.201848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.201848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.202324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:46.204342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:46.290428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e859d826-0107-4eb8-80ef-fe326e51c812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d3c3546f6a1448485d43619cbfb4322'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:46.297433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 976/1928] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:45 2022] DELETE /identity/v3/users/9d3c3546f6a1448485d43619cbfb4322 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:46.301686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.301973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.302251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.321572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:46.323145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:46.353723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 953/1929] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:46 2022] POST /identity/v3/users => generated 300 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:46.358023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:46.359800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:46.372262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea977-61b3-4b27-97a1-ba7777f0b0ee admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:46.386175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 977/1930] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:46.396807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:46.398207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:46.431316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:46.435520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:46.436163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.436451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.436762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.437477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:46.439168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:46.450077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 954/1931] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:46 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:46.455735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:46.465609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3ea24-fa9e-4bc6-b997-ca809c7d98ce None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:46.469537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204619e3-25be-4efd-8450-b71bbeff3d1a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:46.471355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:46.471988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.472275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.472592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.474176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:46.474840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.475119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.475393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] PATH_INFO: `/v3/projects/19c3fe3b179444188e8d29b33514fe2e/users/47be445e817842319feed53d6a485ac9/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.482291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:46.483797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:46.492803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c3fe3b179444188e8d29b33514fe2e, user_id=47be445e817842319feed53d6a485ac9, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:46.493781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 978/1932] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:46.495566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:46.633952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 955/1933] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:46 2022] PUT /identity/v3/projects/19c3fe3b179444188e8d29b33514fe2e/users/47be445e817842319feed53d6a485ac9/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:46.638319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:46.649292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8fa79-0f45-409f-9996-695b4c179cbd service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:46.708026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312ba4-a89a-45d2-bf29-5b6822d0ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:46.708769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312ba4-a89a-45d2-bf29-5b6822d0ef67 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.709127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312ba4-a89a-45d2-bf29-5b6822d0ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.714624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312ba4-a89a-45d2-bf29-5b6822d0ef67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.714624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312ba4-a89a-45d2-bf29-5b6822d0ef67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:46.714624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312ba4-a89a-45d2-bf29-5b6822d0ef67 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:46.726029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 979/1934] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:46 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:46.727290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:46.738713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9765556-9e32-44d8-876b-16b009d777fb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:46.744218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:46.744819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.745096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.745375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] PATH_INFO: `/v3/projects/19c3fe3b179444188e8d29b33514fe2e/users/47be445e817842319feed53d6a485ac9/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.748743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c3fe3b179444188e8d29b33514fe2e, user_id=47be445e817842319feed53d6a485ac9, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:46.751720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:46.754311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:46.768670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 956/1935] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:46 2022] PUT /identity/v3/projects/19c3fe3b179444188e8d29b33514fe2e/users/47be445e817842319feed53d6a485ac9/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:46.783347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.783853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.784345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:46.820424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] MFA Rules not processed for user `47be445e817842319feed53d6a485ac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:46.821708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:46.901546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 980/1936] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:46.914667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:46.916453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:46.980872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1b4bf-d5b2-4986-b28b-70ec3f64c081 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:46.985293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:46.986056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:46.986343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:46.986624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:47.001910 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:47.002074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:47.002544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:47.002630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:47.014723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 957/1937] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:47.088593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:47.097448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:47.110266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b862a49-edba-4fa3-b681-0f440b98d8bf service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:47.113478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299576b4-8aa1-4620-a5f7-76836f18eb92 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:47.115633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:47.116361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:47.116833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:47.117280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] PATH_INFO: `/v3/users/200b7dc83e624717a380ccddbac8ea72` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:47.118570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:47.118699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=200b7dc83e624717a380ccddbac8ea72)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:47.127099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:47.127099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:47.127099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] PATH_INFO: `/v3/projects/d92ba80d86c54035a3e58376409e9cae` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:47.127668 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:47.127759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92ba80d86c54035a3e58376409e9cae)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:47.127759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:47.385993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92ba80d86c54035a3e58376409e9cae', 'request_id': 'req-ff11a5c7-a459-4213-870c-2145e70b449a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.386229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200b7dc83e624717a380ccddbac8ea72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.393066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92ba80d86c54035a3e58376409e9cae', 'request_id': 'req-ff11a5c7-a459-4213-870c-2145e70b449a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.394464 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200b7dc83e624717a380ccddbac8ea72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.395186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200b7dc83e624717a380ccddbac8ea72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.398957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92ba80d86c54035a3e58376409e9cae', 'request_id': 'req-ff11a5c7-a459-4213-870c-2145e70b449a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.473007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200b7dc83e624717a380ccddbac8ea72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.473752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e58102d-e18a-44a4-b8a7-9adea0c99a66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200b7dc83e624717a380ccddbac8ea72'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.492520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 958/1938] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:47 2022] DELETE /identity/v3/users/200b7dc83e624717a380ccddbac8ea72 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:47.497471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:47.497929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:47.498215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:47.517851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:47.519067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:47.535481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] The token cache is being invalidate because project d92ba80d86c54035a3e58376409e9cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:47.535857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.536517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.538060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 981/1939] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:46 2022] DELETE /identity/v3/projects/d92ba80d86c54035a3e58376409e9cae => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:47.542055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:47.543353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:47.583878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 959/1940] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:47.594130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:47.596340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:47.606195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:47.612374 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11a5c7-a459-4213-870c-2145e70b449a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:47.617131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:47.617786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:47.618071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:47.618353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] PATH_INFO: `/v3/users/38ab66b7ee67472690f9d145268af796` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:47.619525 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ab66b7ee67472690f9d145268af796)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:47.621540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:47.658906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5673d-3da4-419e-b54c-95962eed3091 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:47.663041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:47.663660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:47.663944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:47.664221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:47.674193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:47.675631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:47.686711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 960/1941] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:47.860639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ab66b7ee67472690f9d145268af796'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.861293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ab66b7ee67472690f9d145268af796'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.868343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ab66b7ee67472690f9d145268af796'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.883498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ab66b7ee67472690f9d145268af796'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.981420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ab66b7ee67472690f9d145268af796'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:47.983349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:47.988817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 982/1942] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:47 2022] DELETE /identity/v3/users/38ab66b7ee67472690f9d145268af796 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:48.132365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:48.133955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:48.150149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7bed-9f55-4736-b314-30c880ffa979 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:48.221614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:48.222282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:48.222571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:48.222851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] PATH_INFO: `/v3/projects/391ac096d47d4713a72526824fcf398c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:48.224034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=391ac096d47d4713a72526824fcf398c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:48.226041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:48.355348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391ac096d47d4713a72526824fcf398c', 'request_id': 'req-b6189c58-b656-42f3-bb54-0090d66457a7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.433605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391ac096d47d4713a72526824fcf398c', 'request_id': 'req-b6189c58-b656-42f3-bb54-0090d66457a7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.440765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391ac096d47d4713a72526824fcf398c', 'request_id': 'req-b6189c58-b656-42f3-bb54-0090d66457a7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.459930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] The token cache is being invalidate because project 391ac096d47d4713a72526824fcf398c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:48.460225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.461180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.465830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 961/1943] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:48 2022] DELETE /identity/v3/projects/391ac096d47d4713a72526824fcf398c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:48.468315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:48.469947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:48.484298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1ea36-8504-4857-87a0-4c92a052427e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:48.547548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:48.548207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:48.548517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:48.548792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] PATH_INFO: `/v3/users/47be445e817842319feed53d6a485ac9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:48.550004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47be445e817842319feed53d6a485ac9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:48.554321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:48.579929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:48.581351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:48.594247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6189c58-b656-42f3-bb54-0090d66457a7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:48.599073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:48.602034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:48.602034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:48.602034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] PATH_INFO: `/v3/projects/e46b3cca61ed469e92e6a629c51852fd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:48.602034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46b3cca61ed469e92e6a629c51852fd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:48.603466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:48.782121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:48.790708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47be445e817842319feed53d6a485ac9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.791002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46b3cca61ed469e92e6a629c51852fd', 'request_id': 'req-c28adbd4-258c-4434-af23-500d028241bb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.791456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47be445e817842319feed53d6a485ac9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.800639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47be445e817842319feed53d6a485ac9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.816353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47be445e817842319feed53d6a485ac9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.889970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46b3cca61ed469e92e6a629c51852fd', 'request_id': 'req-c28adbd4-258c-4434-af23-500d028241bb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.895893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46b3cca61ed469e92e6a629c51852fd', 'request_id': 'req-c28adbd4-258c-4434-af23-500d028241bb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.908223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] The token cache is being invalidate because project e46b3cca61ed469e92e6a629c51852fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:48.908547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.909218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c28adbd4-258c-4434-af23-500d028241bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.910743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 962/1944] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:48 2022] DELETE /identity/v3/projects/e46b3cca61ed469e92e6a629c51852fd => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:48.916357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a709f8-ec50-4f8a-b072-bd30dd3b250e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:48.916691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a709f8-ec50-4f8a-b072-bd30dd3b250e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:48.916974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a709f8-ec50-4f8a-b072-bd30dd3b250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:48.938462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-60a709f8-ec50-4f8a-b072-bd30dd3b250e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:48.939265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-60a709f8-ec50-4f8a-b072-bd30dd3b250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:48.992751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-886d869d-068e-4b79-be09-0a3fda589049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47be445e817842319feed53d6a485ac9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:48.999778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 983/1945] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:48 2022] DELETE /identity/v3/users/47be445e817842319feed53d6a485ac9 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:49.004253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 963/1946] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:49.013266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c9bcc-786f-4fef-b9e6-36cd6732a5b5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.013646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c9bcc-786f-4fef-b9e6-36cd6732a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.013999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c9bcc-786f-4fef-b9e6-36cd6732a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.035106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e65c9bcc-786f-4fef-b9e6-36cd6732a5b5 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:49.036423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e65c9bcc-786f-4fef-b9e6-36cd6732a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:49.102705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 984/1947] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:49.107883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a709f8-ec50-4f8a-b072-bd30dd3b250e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.118301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a709f8-ec50-4f8a-b072-bd30dd3b250e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.123176 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.123860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.124137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.124413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.125236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:49.125561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.127168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.140511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 964/1948] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:49.145053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c9bcc-786f-4fef-b9e6-36cd6732a5b5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.155672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c9bcc-786f-4fef-b9e6-36cd6732a5b5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.160726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.161358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.161644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.161922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.162655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.164575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.184808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:49.293068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 985/1949] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:49 2022] POST /identity/v3/projects => generated 384 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:49.298108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.324969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342dfb-299d-4dfd-bb30-7462107c36b2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.393507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.394082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.394384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.394693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.395404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.397397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.471719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.485627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:49.503015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a513b39-0b30-4bb5-b08b-3aa741908778 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.510251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.511542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.511861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.512214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] PATH_INFO: `/v3/projects/19c3fe3b179444188e8d29b33514fe2e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.514202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c3fe3b179444188e8d29b33514fe2e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.517358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.541406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 965/1950] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:49 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:49.546656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.557697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25104-304a-4707-b73a-3472764ce621 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.563639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.564337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.564674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.565041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.566354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.567749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.583911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 966/1951] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:49 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:49.585237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.599103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d4254-9596-4a28-81e0-aea7b1b933d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.612010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.612010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.612010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.612010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.612544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.612544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.631348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 967/1952] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:49 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:49.634477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.646628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd6fa7-62b8-49cf-870c-523fd5396e5a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.654070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283752d-e6e4-4031-bd84-9b61a1511224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.654998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283752d-e6e4-4031-bd84-9b61a1511224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.655638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283752d-e6e4-4031-bd84-9b61a1511224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.660395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283752d-e6e4-4031-bd84-9b61a1511224 admin admin] PATH_INFO: `/v3/projects/dac818cf11c14f33acd17751b11203f9/users/d2c082b4f4e94176a5d79e162f5d66ce/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.680701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c3fe3b179444188e8d29b33514fe2e', 'request_id': 'req-6da6b2ac-8f39-444b-be25-266a3811a8c8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:49.687746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c3fe3b179444188e8d29b33514fe2e', 'request_id': 'req-6da6b2ac-8f39-444b-be25-266a3811a8c8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:49.692039 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283752d-e6e4-4031-bd84-9b61a1511224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac818cf11c14f33acd17751b11203f9, user_id=d2c082b4f4e94176a5d79e162f5d66ce, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.693792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c3fe3b179444188e8d29b33514fe2e', 'request_id': 'req-6da6b2ac-8f39-444b-be25-266a3811a8c8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:49.696025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283752d-e6e4-4031-bd84-9b61a1511224 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.771168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] The token cache is being invalidate because project 19c3fe3b179444188e8d29b33514fe2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:49.772625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:49.772625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6da6b2ac-8f39-444b-be25-266a3811a8c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:49.773767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 986/1953] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:49 2022] DELETE /identity/v3/projects/19c3fe3b179444188e8d29b33514fe2e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:49.779222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69e684-896c-42df-af43-6442c2724831 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.779522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69e684-896c-42df-af43-6442c2724831 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.779808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69e684-896c-42df-af43-6442c2724831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.802291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 968/1954] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:49 2022] PUT /identity/v3/projects/dac818cf11c14f33acd17751b11203f9/users/d2c082b4f4e94176a5d79e162f5d66ce/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:49.814600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.815044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6f69e684-896c-42df-af43-6442c2724831 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:49.815165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.815498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.816999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6f69e684-896c-42df-af43-6442c2724831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:49.860955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] MFA Rules not processed for user `d2c082b4f4e94176a5d79e162f5d66ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:49.862234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:49.904579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 987/1955] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:49.911277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911430e-f52d-4dda-a840-b08f8adc1ddc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.911567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911430e-f52d-4dda-a840-b08f8adc1ddc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.911840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911430e-f52d-4dda-a840-b08f8adc1ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.933542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6911430e-f52d-4dda-a840-b08f8adc1ddc None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:49.934808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6911430e-f52d-4dda-a840-b08f8adc1ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:49.941528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:49.944435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 988/1956] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:49.949393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911430e-f52d-4dda-a840-b08f8adc1ddc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.954015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 969/1957] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:49.959216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911430e-f52d-4dda-a840-b08f8adc1ddc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:49.964695 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:49.965326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:49.965463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.966478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:49.966770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:49.967600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:49.967711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:49.967940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:49.969686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:49.987384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 989/1958] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:36:49.988599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:49.999257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c295a27-1ef6-40db-b842-2ab7f908c339 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.005984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.006630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.006949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.007226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.008197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.010190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.035735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274b482-9d41-40e8-9c10-9d81d3563eb4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.040815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.041519 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.041805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.042084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.052341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.053923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.064229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 970/1959] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:50.155435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 990/1960] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:49 2022] POST /identity/v3/projects => generated 380 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:50.159943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.172309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae042c-c4e6-49c1-88af-78688889139d service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.225651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.227351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:50.238034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.238796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.239085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.242103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.242103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.242103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.292250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92accb-9c87-40d4-a197-db7fa5ca0564 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.298251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.298902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.299200 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.299566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] PATH_INFO: `/v3/users/aad4757e1c5548a28531307a90351c00` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.300866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad4757e1c5548a28531307a90351c00)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.302750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.352626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 971/1961] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:50 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:50.357221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.367169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05264c-26c0-4414-9437-55d6d80161ba admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.373256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.373797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.374084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.374361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.375137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.376727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.376854 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:50.388074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 972/1962] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:50 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:50.392288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.403859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692730-47ad-43ed-938f-1f95ac1591ac admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.413296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.413296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.413296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.413296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.413666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.413750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.425408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 973/1963] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:50 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:50.429516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.440110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b1efc-e421-46ba-9b41-9da098c6cfb1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.446609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3f75f-ad78-4019-ad5a-bd4d5e9de99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.447546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3f75f-ad78-4019-ad5a-bd4d5e9de99e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.447832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3f75f-ad78-4019-ad5a-bd4d5e9de99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.448116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3f75f-ad78-4019-ad5a-bd4d5e9de99e admin admin] PATH_INFO: `/v3/projects/48133a3d94214dff8aa2088e11d259d2/users/0ef9a1f304184d5cbc95d7d69ef447d7/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.470773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3f75f-ad78-4019-ad5a-bd4d5e9de99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=48133a3d94214dff8aa2088e11d259d2, user_id=0ef9a1f304184d5cbc95d7d69ef447d7, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.471381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3f75f-ad78-4019-ad5a-bd4d5e9de99e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.532531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad4757e1c5548a28531307a90351c00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:50.533258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad4757e1c5548a28531307a90351c00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:50.539942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad4757e1c5548a28531307a90351c00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:50.553311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 974/1964] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:50 2022] PUT /identity/v3/projects/48133a3d94214dff8aa2088e11d259d2/users/0ef9a1f304184d5cbc95d7d69ef447d7/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:50.553836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad4757e1c5548a28531307a90351c00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:50.563992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a8724-406b-4a96-9d52-e9ce62d25f83 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.564310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a8724-406b-4a96-9d52-e9ce62d25f83 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.564646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a8724-406b-4a96-9d52-e9ce62d25f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.594136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e49a8724-406b-4a96-9d52-e9ce62d25f83 None None] MFA Rules not processed for user `0ef9a1f304184d5cbc95d7d69ef447d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:50.595392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e49a8724-406b-4a96-9d52-e9ce62d25f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:50.660883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 975/1965] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:50.665526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a8724-406b-4a96-9d52-e9ce62d25f83 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.689809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e597002-dca3-4637-b2d6-6af5125a1b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad4757e1c5548a28531307a90351c00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:50.698202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 991/1966] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:50 2022] DELETE /identity/v3/users/aad4757e1c5548a28531307a90351c00 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:50.702990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.703284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.704234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.724328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:50.725724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:50.739058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a8724-406b-4a96-9d52-e9ce62d25f83 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.791820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 992/1967] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:50.801631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.802269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.802560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.802840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.803571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.804982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.805547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.806422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:50.873098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02487e-1f48-4c59-87d0-d8d0cfa23e25 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:50.878373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:50.878939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:50.879293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:50.879493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:50.891029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:50.892532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:50.903041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 993/1968] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:50.917322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 976/1969] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:50 2022] POST /identity/v3/projects => generated 380 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:50.923664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:50.925520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:50.941689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8753902-5026-4d2c-89b4-c043628e8d5e service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.022442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.023177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.023543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.023834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.024573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.027417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.153234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 994/1970] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:50 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:51.159393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.171506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126385a8-8bc2-4cf2-9449-92818af3f263 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.178041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.179500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.179500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.179500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.180778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.182551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.200521 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 977/1971] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:51 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:51.205918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.219336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59170441-e929-4cb6-b872-f26a7d184017 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.226103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.226937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.227368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.227709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.228717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.230586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.246428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 995/1972] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:51 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:51.255258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.266173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0522da9-4bc1-4e7c-8d3e-52f4dc9314eb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.272595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:51.273756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c6756-4acd-42af-b55e-568c377b20b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.274474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c6756-4acd-42af-b55e-568c377b20b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.274749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c6756-4acd-42af-b55e-568c377b20b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.275032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c6756-4acd-42af-b55e-568c377b20b8 admin admin] PATH_INFO: `/v3/projects/be2bb3de9789417aaa0337b128ceda9c/users/4bfb8d76e1294f6dbd966c2c6a8d402b/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.290727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c6756-4acd-42af-b55e-568c377b20b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2bb3de9789417aaa0337b128ceda9c, user_id=4bfb8d76e1294f6dbd966c2c6a8d402b, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.293453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c6756-4acd-42af-b55e-568c377b20b8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.392717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:51.399684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.401728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:51.419582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb187eb-4f8f-454d-8c1a-98273b751944 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.424937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 978/1973] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:51 2022] PUT /identity/v3/projects/be2bb3de9789417aaa0337b128ceda9c/users/4bfb8d76e1294f6dbd966c2c6a8d402b/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:51.436543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a048d07-2a9a-4f1d-ad0d-a6de62edb01d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.436834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a048d07-2a9a-4f1d-ad0d-a6de62edb01d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.437114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a048d07-2a9a-4f1d-ad0d-a6de62edb01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.469863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4a048d07-2a9a-4f1d-ad0d-a6de62edb01d None None] MFA Rules not processed for user `4bfb8d76e1294f6dbd966c2c6a8d402b`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:51.471238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4a048d07-2a9a-4f1d-ad0d-a6de62edb01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:51.486961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.487943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.488219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.488555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] PATH_INFO: `/v3/projects/0c186c1ba3bb4c9f9681c7c04ed20a03` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.489800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c186c1ba3bb4c9f9681c7c04ed20a03)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.491732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.538025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 979/1974] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:51.547964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a048d07-2a9a-4f1d-ad0d-a6de62edb01d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.558395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a048d07-2a9a-4f1d-ad0d-a6de62edb01d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.563551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.564158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.564441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.564761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.565422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.567364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.617487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c186c1ba3bb4c9f9681c7c04ed20a03', 'request_id': 'req-ad052ae6-c077-46a6-8de1-a154bc2a54c8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:51.623639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c186c1ba3bb4c9f9681c7c04ed20a03', 'request_id': 'req-ad052ae6-c077-46a6-8de1-a154bc2a54c8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:51.629395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c186c1ba3bb4c9f9681c7c04ed20a03', 'request_id': 'req-ad052ae6-c077-46a6-8de1-a154bc2a54c8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:51.633200 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:51.704187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 980/1975] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:51 2022] POST /identity/v3/projects => generated 380 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:51.708064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.726030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] The token cache is being invalidate because project 0c186c1ba3bb4c9f9681c7c04ed20a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:51.726697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:51.727110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:51.728900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 996/1976] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:51 2022] DELETE /identity/v3/projects/0c186c1ba3bb4c9f9681c7c04ed20a03 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:51.733223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.735133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:51.775987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18563525-4cba-4507-b2f7-86e30e3a1c8c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.782447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.785913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.785913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.785913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.785913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.786417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.807652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad052ae6-c077-46a6-8de1-a154bc2a54c8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.813584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.822310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.822310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.822310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] PATH_INFO: `/v3/users/7ff20911913546648108cd057e43fd1e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.822310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff20911913546648108cd057e43fd1e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.822310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.892488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:51.897995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 981/1977] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:51 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:51.901894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.903683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:51.917164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896193ac-169a-45a9-bb86-5070aeaedb37 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.930949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.930949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.930949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.930949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.932187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.932187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.942722 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 982/1978] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:51 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:51.943694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:51.958373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1c036-4678-4e35-8c99-d6f96a950ea4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:51.963289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:51.964040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:51.964260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:51.964582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] PATH_INFO: `/v3/projects/a935c1c8a2fe4509ac717df4bdea4865/users/fd15ebc747e3455c85a2ee803c6735d6/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:51.980389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a935c1c8a2fe4509ac717df4bdea4865, user_id=fd15ebc747e3455c85a2ee803c6735d6, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:51.983318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:51.992573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:52.081790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 983/1979] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:51 2022] PUT /identity/v3/projects/a935c1c8a2fe4509ac717df4bdea4865/users/fd15ebc747e3455c85a2ee803c6735d6/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:52.082457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:52.086888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff20911913546648108cd057e43fd1e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:52.087629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff20911913546648108cd057e43fd1e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:52.095828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff20911913546648108cd057e43fd1e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:52.109469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff20911913546648108cd057e43fd1e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:52.159932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3ad4c-286a-44e2-94af-b9ed68c813a3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:52.166870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026dde1-8aff-41ef-9703-d8bdbe3abe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:52.167516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026dde1-8aff-41ef-9703-d8bdbe3abe39 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:52.167805 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026dde1-8aff-41ef-9703-d8bdbe3abe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:52.168096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026dde1-8aff-41ef-9703-d8bdbe3abe39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:52.168892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026dde1-8aff-41ef-9703-d8bdbe3abe39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:52.170328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff20911913546648108cd057e43fd1e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:52.170678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026dde1-8aff-41ef-9703-d8bdbe3abe39 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:52.181960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 997/1980] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:51 2022] DELETE /identity/v3/users/7ff20911913546648108cd057e43fd1e => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:52.190465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 984/1981] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:52 2022] GET /identity/v3/roles => generated 1427 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:52.194416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:52.197378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:52.301771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70fe65-3e80-4ca1-9593-eaf74b962fc1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:52.306909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:52.308793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:52.309149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:52.309508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] PATH_INFO: `/v3/projects/a935c1c8a2fe4509ac717df4bdea4865/users/fd15ebc747e3455c85a2ee803c6735d6/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:52.312380 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a935c1c8a2fe4509ac717df4bdea4865, user_id=fd15ebc747e3455c85a2ee803c6735d6, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:52.316081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:52.327995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 998/1982] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:52 2022] PUT /identity/v3/projects/a935c1c8a2fe4509ac717df4bdea4865/users/fd15ebc747e3455c85a2ee803c6735d6/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:52.337990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:52.338452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:52.338743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:52.369957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] MFA Rules not processed for user `fd15ebc747e3455c85a2ee803c6735d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:52.371841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:52.470399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 985/1983] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:52.478927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:52.483172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:52.552397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3e58-f107-4e5c-8e2a-4b6a81c2e8d2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:52.556291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:52.557083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:52.557533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:52.557891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:52.570277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:52.572170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:52.582669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 999/1984] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:52.680371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:52.682067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:52.773742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99dce-5850-4182-a93b-f8b9acd4f10b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:52.781098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:52.781865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:52.782195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:52.782518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] PATH_INFO: `/v3/projects/379a187a97cd4a6eafcd478b2fd4972d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:52.783900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=379a187a97cd4a6eafcd478b2fd4972d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:52.786211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:52.788961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:52.821659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:52.904622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '379a187a97cd4a6eafcd478b2fd4972d', 'request_id': 'req-d02a4328-7958-492e-8d11-9119610558c5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:52.907568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:52.909190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:52.923566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc88858-acb5-47a0-827f-db990cfc10fd service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:52.981457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:52.982151 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:52.982505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:52.982871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] PATH_INFO: `/v3/users/a3fbc326e3134e02bc722bdd54ce5a32` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:52.984366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3fbc326e3134e02bc722bdd54ce5a32)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:52.986410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.063468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '379a187a97cd4a6eafcd478b2fd4972d', 'request_id': 'req-d02a4328-7958-492e-8d11-9119610558c5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.071683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '379a187a97cd4a6eafcd478b2fd4972d', 'request_id': 'req-d02a4328-7958-492e-8d11-9119610558c5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.085695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] The token cache is being invalidate because project 379a187a97cd4a6eafcd478b2fd4972d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:53.085988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.086663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.088197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 986/1985] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:52 2022] DELETE /identity/v3/projects/379a187a97cd4a6eafcd478b2fd4972d => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:53.089006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.090313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.148105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a4328-7958-492e-8d11-9119610558c5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.151872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482a0dc-1f10-4877-abf4-b90714f8d057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.152517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482a0dc-1f10-4877-abf4-b90714f8d057 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.152808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482a0dc-1f10-4877-abf4-b90714f8d057 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.153086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482a0dc-1f10-4877-abf4-b90714f8d057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.155674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9482a0dc-1f10-4877-abf4-b90714f8d057 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.207004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3fbc326e3134e02bc722bdd54ce5a32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.207738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3fbc326e3134e02bc722bdd54ce5a32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.215304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3fbc326e3134e02bc722bdd54ce5a32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.219624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482a0dc-1f10-4877-abf4-b90714f8d057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.221172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482a0dc-1f10-4877-abf4-b90714f8d057 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.229018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3fbc326e3134e02bc722bdd54ce5a32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.230472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 987/1986] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:53.231732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d7bf2-8ef7-42be-a246-7bf43e14faa1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.232137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d7bf2-8ef7-42be-a246-7bf43e14faa1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.232415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d7bf2-8ef7-42be-a246-7bf43e14faa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.252077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f15d7bf2-8ef7-42be-a246-7bf43e14faa1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:53.253330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f15d7bf2-8ef7-42be-a246-7bf43e14faa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.264072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 988/1987] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:53.271973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.272316 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.272561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.294898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:53.296418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.304501 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 989/1988] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:53.305315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.306527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.318726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26198241-414e-4820-829f-dc0123f4bae1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.323759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.324417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.324730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.325014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] PATH_INFO: `/v3/users/d2c082b4f4e94176a5d79e162f5d66ce` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.326981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c082b4f4e94176a5d79e162f5d66ce)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.329024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.329607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3fbc326e3134e02bc722bdd54ce5a32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.337817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1000/1989] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:52 2022] DELETE /identity/v3/users/a3fbc326e3134e02bc722bdd54ce5a32 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:53.340015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.343829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.410661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa6244-1cc4-4262-afd9-1d9742341b74 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.415923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af85a3-6104-43f7-94fe-3536a5629ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.417011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af85a3-6104-43f7-94fe-3536a5629ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.417400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af85a3-6104-43f7-94fe-3536a5629ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.417703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af85a3-6104-43f7-94fe-3536a5629ad0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.418486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af85a3-6104-43f7-94fe-3536a5629ad0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:53.418790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af85a3-6104-43f7-94fe-3536a5629ad0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.420693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af85a3-6104-43f7-94fe-3536a5629ad0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.433042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1001/1990] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:53.434312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3aa7f-7e26-4183-a011-263a64bd03c8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.434602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3aa7f-7e26-4183-a011-263a64bd03c8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.434881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3aa7f-7e26-4183-a011-263a64bd03c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.456592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1ce3aa7f-7e26-4183-a011-263a64bd03c8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:53.458163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1ce3aa7f-7e26-4183-a011-263a64bd03c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.467720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1002/1991] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:53.468984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3aa7f-7e26-4183-a011-263a64bd03c8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.480249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3aa7f-7e26-4183-a011-263a64bd03c8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.485719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.486427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.486787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.487076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.487760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.489791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.574592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c082b4f4e94176a5d79e162f5d66ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.582770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c082b4f4e94176a5d79e162f5d66ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.583499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c082b4f4e94176a5d79e162f5d66ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.589546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1003/1992] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:53 2022] POST /identity/v3/projects => generated 384 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:53.590387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.592037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.650119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01df2d9-af56-475d-ab00-8c065d0b158f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.653859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330cfd6-ccd9-4569-9c86-d52c9584ddef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.654991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330cfd6-ccd9-4569-9c86-d52c9584ddef service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.655279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330cfd6-ccd9-4569-9c86-d52c9584ddef service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.655558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330cfd6-ccd9-4569-9c86-d52c9584ddef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.657485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2330cfd6-ccd9-4569-9c86-d52c9584ddef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.689594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c082b4f4e94176a5d79e162f5d66ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.690784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c082b4f4e94176a5d79e162f5d66ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:53.712609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 990/1993] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:53 2022] DELETE /identity/v3/users/d2c082b4f4e94176a5d79e162f5d66ce => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:53.715336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.715336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.728871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330cfd6-ccd9-4569-9c86-d52c9584ddef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.730327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330cfd6-ccd9-4569-9c86-d52c9584ddef service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.740836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1004/1994] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:53.743258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ba2b1-c740-4308-9926-b36200ff12f2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.743801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ba2b1-c740-4308-9926-b36200ff12f2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.744091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ba2b1-c740-4308-9926-b36200ff12f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.767231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-807ba2b1-c740-4308-9926-b36200ff12f2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:53.768658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-807ba2b1-c740-4308-9926-b36200ff12f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:53.792251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56221fb-67bb-4810-8f2e-4e5ed49f0f69 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.800111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.800871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.801162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.801444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.802134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.804221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.837963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1005/1995] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:53.850424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ba2b1-c740-4308-9926-b36200ff12f2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.901727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 991/1996] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:53 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:53.905919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.910007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ba2b1-c740-4308-9926-b36200ff12f2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.914024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.914670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.914948 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.915252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.915523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20efa6f-a963-4bf6-9136-46caec18a330 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.920793 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.921406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.921693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.921973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.922703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.924317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.925855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.927297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.939717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1006/1997] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:53.939936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:53.940511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 992/1998] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:53 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:53.943175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.955504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75f7dd-b992-4ad4-8284-fb6c3241c7ad service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:53.962449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb48f96-e255-4cde-8423-aac45b1a5237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:53.963541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb48f96-e255-4cde-8423-aac45b1a5237 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:53.964000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb48f96-e255-4cde-8423-aac45b1a5237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:53.964887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb48f96-e255-4cde-8423-aac45b1a5237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:53.965929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb48f96-e255-4cde-8423-aac45b1a5237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:53.968847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb48f96-e255-4cde-8423-aac45b1a5237 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:53.981095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1007/1999] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:53 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:53.987506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:53.998794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a863d-69d3-42f8-aca4-d3fc30349825 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:54.003643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:54.006057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.006057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.006057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] PATH_INFO: `/v3/projects/5f908335d06a4da4922bbdde9b8cc1ce/users/3fff6a0c1fe84662871a55e147960bd6/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.034241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f908335d06a4da4922bbdde9b8cc1ce, user_id=3fff6a0c1fe84662871a55e147960bd6, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:54.037073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:54.108307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 993/2000] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:53 2022] PUT /identity/v3/projects/5f908335d06a4da4922bbdde9b8cc1ce/users/3fff6a0c1fe84662871a55e147960bd6/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:54.120200 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.120610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.120964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.172059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] MFA Rules not processed for user `3fff6a0c1fe84662871a55e147960bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:54.173377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:54.248495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1008/2001] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:54.253485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:54.316623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:54.318520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:54.334803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d268d-de47-41dd-bbe3-179e3dadb851 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:54.340198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:54.340862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.341156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.341484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.342218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:54.344232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:54.389664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b01560-3b1e-4906-ae57-6c41ab066edd None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:54.394601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:54.395541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.395812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.396091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] PATH_INFO: `/v3/projects/3dbbcdfca4c24dcf875a3e28a5d4953b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.397334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dbbcdfca4c24dcf875a3e28a5d4953b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:54.399279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:54.478480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 994/2002] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:54 2022] POST /identity/v3/projects => generated 382 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:54.478480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:54.521403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dbbcdfca4c24dcf875a3e28a5d4953b', 'request_id': 'req-39795072-9be5-4665-b4e1-3f3961cbf0bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.527845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dbbcdfca4c24dcf875a3e28a5d4953b', 'request_id': 'req-39795072-9be5-4665-b4e1-3f3961cbf0bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.536145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dbbcdfca4c24dcf875a3e28a5d4953b', 'request_id': 'req-39795072-9be5-4665-b4e1-3f3961cbf0bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.557512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38bbde-13e0-4955-9a7f-33ede7279c2b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:54.615573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] The token cache is being invalidate because project 3dbbcdfca4c24dcf875a3e28a5d4953b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:54.615870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.616540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.616764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:54.617772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.617772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.618022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.618215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1009/2003] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:54 2022] DELETE /identity/v3/projects/3dbbcdfca4c24dcf875a3e28a5d4953b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:54.618618 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:54.619069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:54.620389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:54.620763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:54.634424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39795072-9be5-4665-b4e1-3f3961cbf0bd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:54.637551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:54.641696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:54.642346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.642629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.642939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] PATH_INFO: `/v3/projects/dac818cf11c14f33acd17751b11203f9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.644094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac818cf11c14f33acd17751b11203f9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:54.663408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:54.746515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 995/2004] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:54 2022] POST /identity/v3/users => generated 298 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:54.747404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:54.749083 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:54.761172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3000d6-8fe2-4cf0-ac37-a98acc5fcadc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:54.761482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac818cf11c14f33acd17751b11203f9', 'request_id': 'req-2fc3520d-bab0-4f87-928d-51961d2fd721'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.768955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac818cf11c14f33acd17751b11203f9', 'request_id': 'req-2fc3520d-bab0-4f87-928d-51961d2fd721'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.775267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac818cf11c14f33acd17751b11203f9', 'request_id': 'req-2fc3520d-bab0-4f87-928d-51961d2fd721'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.822687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:54.823350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.823647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.823906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] PATH_INFO: `/v3/users/c691e410117f400cac136af5ac125ea2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.825217 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c691e410117f400cac136af5ac125ea2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:54.828536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:54.914210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] The token cache is being invalidate because project dac818cf11c14f33acd17751b11203f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:54.914737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.915081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:54.916709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1010/2005] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:54 2022] DELETE /identity/v3/projects/dac818cf11c14f33acd17751b11203f9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:54.917474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:54.918753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:54.931618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3520d-bab0-4f87-928d-51961d2fd721 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:54.937097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef9e81-b58f-4e98-996b-f72f7e898dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:54.937669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9e81-b58f-4e98-996b-f72f7e898dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.937958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9e81-b58f-4e98-996b-f72f7e898dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.938237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9e81-b58f-4e98-996b-f72f7e898dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.938946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef9e81-b58f-4e98-996b-f72f7e898dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:54.940610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef9e81-b58f-4e98-996b-f72f7e898dfe admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:54.952733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1011/2006] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:54 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:54.954064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83064bfe-72b2-42e5-9cd4-d832d366fb8b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:54.954363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83064bfe-72b2-42e5-9cd4-d832d366fb8b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:54.954642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83064bfe-72b2-42e5-9cd4-d832d366fb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:54.974078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-83064bfe-72b2-42e5-9cd4-d832d366fb8b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:54.975329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-83064bfe-72b2-42e5-9cd4-d832d366fb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:54.985676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1012/2007] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:54.986622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83064bfe-72b2-42e5-9cd4-d832d366fb8b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:54.995957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83064bfe-72b2-42e5-9cd4-d832d366fb8b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.001400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951338a-cbc7-4c4a-b502-b5e04562add6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.002056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951338a-cbc7-4c4a-b502-b5e04562add6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.002344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951338a-cbc7-4c4a-b502-b5e04562add6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.002626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951338a-cbc7-4c4a-b502-b5e04562add6 admin admin] PATH_INFO: `/v3/projects/798c9d548f784a8d9c475bf8f96918c9/users/7dc228f53dd04005973c1f694c224633/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.017755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951338a-cbc7-4c4a-b502-b5e04562add6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798c9d548f784a8d9c475bf8f96918c9, user_id=7dc228f53dd04005973c1f694c224633, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.020633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951338a-cbc7-4c4a-b502-b5e04562add6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.189273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c691e410117f400cac136af5ac125ea2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:55.196396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c691e410117f400cac136af5ac125ea2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:55.197404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c691e410117f400cac136af5ac125ea2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:55.286021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1013/2008] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:54 2022] PUT /identity/v3/projects/798c9d548f784a8d9c475bf8f96918c9/users/7dc228f53dd04005973c1f694c224633/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:55.287634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705837e-ee27-4459-9be7-230a74f34829 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.288142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705837e-ee27-4459-9be7-230a74f34829 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.294955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705837e-ee27-4459-9be7-230a74f34829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.309081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4705837e-ee27-4459-9be7-230a74f34829 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:55.311330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4705837e-ee27-4459-9be7-230a74f34829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:55.323752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c691e410117f400cac136af5ac125ea2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:55.325479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c691e410117f400cac136af5ac125ea2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:55.346098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 996/2009] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:54 2022] DELETE /identity/v3/users/c691e410117f400cac136af5ac125ea2 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:55.349990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.351782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:55.377528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1014/2010] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:55.381520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705837e-ee27-4459-9be7-230a74f34829 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.391669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705837e-ee27-4459-9be7-230a74f34829 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.425972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2ae70-4d1b-4599-99d2-65f44b7c7ff6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.436553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.437224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.437578 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.437923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.438711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.440725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.455652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 997/2011] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:55 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:55.457766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.461689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.462739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.463439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.463836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.464701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:55.465437 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.467059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.471511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750e77e-8a5a-4fbf-bbb1-77c4f66f02b0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.476733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea372ef5-4f32-4cb5-8107-32ade2000c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.477500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372ef5-4f32-4cb5-8107-32ade2000c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.477787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372ef5-4f32-4cb5-8107-32ade2000c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.478074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372ef5-4f32-4cb5-8107-32ade2000c12 admin admin] PATH_INFO: `/v3/projects/798c9d548f784a8d9c475bf8f96918c9/users/7dc228f53dd04005973c1f694c224633/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.480080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1015/2012] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:36:55.480450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea372ef5-4f32-4cb5-8107-32ade2000c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798c9d548f784a8d9c475bf8f96918c9, user_id=7dc228f53dd04005973c1f694c224633, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.484629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea372ef5-4f32-4cb5-8107-32ade2000c12 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.485289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.496181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d2fd-fdf5-4d29-93ce-f31e497ffca9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.506211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 998/2013] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:55 2022] PUT /identity/v3/projects/798c9d548f784a8d9c475bf8f96918c9/users/7dc228f53dd04005973c1f694c224633/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:55.508698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.509066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.509415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.539548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] MFA Rules not processed for user `7dc228f53dd04005973c1f694c224633`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:55.540997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:55.570659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.571339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.571840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.572131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.572951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.576209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.622507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:55.623319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 999/2014] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:55.632378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.633850 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:55.694250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f9be3-b6c2-4f64-9106-1a3b33e8431f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.700521 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.701229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.701510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.701790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.711691 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.713283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.717240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1016/2015] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:55 2022] POST /identity/v3/projects => generated 366 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:55.722121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.726170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1000/2016] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:55.796264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd692b27-5432-4494-8429-56aa8555930e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.804746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.804746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.805022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.805308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.806046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.808772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.890105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.898815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1719a4a-9471-49a3-9434-4ba15a417ad7 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.942449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1017/2017] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:55 2022] POST /identity/v3/users => generated 284 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:36:55.946505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.957619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384711-4920-4e69-af0d-4963508f8931 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:55.959047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.959790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.960261 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.960800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.962627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:55.962783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:55.963590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:55.963936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:55.964284 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:55.965091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:55.967174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:55.978474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1018/2018] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:55 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:55.985420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:55.996355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadca5f6-1d12-4fab-a1df-9c7778cae76d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.003639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.004326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.004661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.005494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.006274 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.008474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.022676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1019/2019] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:55 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:36:56.026890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.033262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.034833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.041113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da0bd1-2df4-4eb2-bf5a-5a92f10aec57 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.045102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1001/2020] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:36:56.046693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419d8f3-012a-4d2c-bd24-07f874bffd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.046894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419d8f3-012a-4d2c-bd24-07f874bffd46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.047184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419d8f3-012a-4d2c-bd24-07f874bffd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.047461 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419d8f3-012a-4d2c-bd24-07f874bffd46 admin admin] PATH_INFO: `/v3/projects/e045fe1c61644fde9dabd25af5f36970/users/2072a421cf924213902987e70da070c7/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.067288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419d8f3-012a-4d2c-bd24-07f874bffd46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e045fe1c61644fde9dabd25af5f36970, user_id=2072a421cf924213902987e70da070c7, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.071123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419d8f3-012a-4d2c-bd24-07f874bffd46 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.082979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.085277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:56.113869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51584a6-e6b8-4471-8d85-36790b1c8869 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.133148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.133148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.133148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.133148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] PATH_INFO: `/v3/projects/3c9bb4ea9c9b48dc96aaa361de12c792` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.133596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9bb4ea9c9b48dc96aaa361de12c792)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.133596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.198627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1020/2021] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:56 2022] PUT /identity/v3/projects/e045fe1c61644fde9dabd25af5f36970/users/2072a421cf924213902987e70da070c7/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:56.209958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaa2fe-a0e3-467f-b803-ded049c3aa69 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.210265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaa2fe-a0e3-467f-b803-ded049c3aa69 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.210534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaa2fe-a0e3-467f-b803-ded049c3aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.238645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cdaaa2fe-a0e3-467f-b803-ded049c3aa69 None None] MFA Rules not processed for user `2072a421cf924213902987e70da070c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:56.239864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cdaaa2fe-a0e3-467f-b803-ded049c3aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:56.262429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9bb4ea9c9b48dc96aaa361de12c792', 'request_id': 'req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:56.315371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1021/2022] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:36:56.335571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9bb4ea9c9b48dc96aaa361de12c792', 'request_id': 'req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:56.345244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9bb4ea9c9b48dc96aaa361de12c792', 'request_id': 'req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:56.359725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] The token cache is being invalidate because project 3c9bb4ea9c9b48dc96aaa361de12c792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:56.360049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:56.360793 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0cc1ca9f-c09c-4a7c-bb03-448d1c136ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:56.362656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1002/2023] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:56 2022] DELETE /identity/v3/projects/3c9bb4ea9c9b48dc96aaa361de12c792 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:36:56.367596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd638065-9726-4470-9988-1263906fabe8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.367995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd638065-9726-4470-9988-1263906fabe8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.368297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd638065-9726-4470-9988-1263906fabe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.376755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:56.394176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fd638065-9726-4470-9988-1263906fabe8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:56.395488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fd638065-9726-4470-9988-1263906fabe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:56.471778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1022/2024] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:56.483183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.483582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.484094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.506024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:56.507372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:56.516767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1003/2025] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:56.521303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd638065-9726-4470-9988-1263906fabe8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.535563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.537545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd638065-9726-4470-9988-1263906fabe8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.541037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:56.542991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.543885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.544211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.545793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.546527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:36:56.547228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.548499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.561853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1023/2026] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:36:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:36:56.566280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.577209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbca89-67de-4098-9492-d928f1c69de1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.582273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.582962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.583255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.583538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.584257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.586224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.609619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1fe60-1be6-4162-99d3-28afa9b835ca None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.615689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.615689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.615689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.615994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.617630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:56.679073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.680672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.685680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1024/2027] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:36:56 2022] POST /identity/v3/projects => generated 386 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:56.689895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.692561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1004/2028] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:56.764373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e53b-a198-4b76-80d7-e171bf9e805f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.769555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.770315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.770467 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.770789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.771481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.773466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.901126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1025/2029] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:36:56 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:36:56.905528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.916221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236aea46-7393-4e92-a47c-aaebbaedb9a2 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.921892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.922422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.922709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.922986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.923687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.925346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.939052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1005/2030] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:56 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:56.943062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.956563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf3f1-202b-4c12-b013-3cdef0635e14 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.961313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5dc18-5139-44e1-8638-7f856147e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.961931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5dc18-5139-44e1-8638-7f856147e8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.962264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5dc18-5139-44e1-8638-7f856147e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.962555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5dc18-5139-44e1-8638-7f856147e8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:56.963643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5dc18-5139-44e1-8638-7f856147e8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:56.964977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5dc18-5139-44e1-8638-7f856147e8a6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:56.975480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1026/2031] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:36:56 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:36:56.980031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:56.991067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27ed56-3dd9-4dc7-ba8f-b2949088a585 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:56.996299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:56.996823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:56.997127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:56.998332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] PATH_INFO: `/v3/projects/915fadab15dc4f249412d83fdfab6f6d/users/113f316bcbb9422283b4ccdc86a56e56/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:57.015977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=915fadab15dc4f249412d83fdfab6f6d, user_id=113f316bcbb9422283b4ccdc86a56e56, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:57.018872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:57.061944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:57.113849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1006/2032] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:36:56 2022] PUT /identity/v3/projects/915fadab15dc4f249412d83fdfab6f6d/users/113f316bcbb9422283b4ccdc86a56e56/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:57.124916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:57.125209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:57.125491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:57.160715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] MFA Rules not processed for user `113f316bcbb9422283b4ccdc86a56e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:57.161908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:57.230826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1027/2033] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:57.241750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:57.243507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:57.303534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e3f4c-e280-4de9-a2b6-b65e0b2111c5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:57.307201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed757a70-06c8-48f3-9f0e-0673781f21d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:57.307789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed757a70-06c8-48f3-9f0e-0673781f21d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:57.308071 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed757a70-06c8-48f3-9f0e-0673781f21d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:57.308346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed757a70-06c8-48f3-9f0e-0673781f21d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:57.317900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed757a70-06c8-48f3-9f0e-0673781f21d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:57.319723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed757a70-06c8-48f3-9f0e-0673781f21d5 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:57.330458 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1007/2034] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:57.444741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:58.085136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:58.089769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:58.176186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde38ed-186f-4a0f-89ce-700ba111e3ff None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:58.181243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:58.181946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:58.182234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:58.182508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] PATH_INFO: `/v3/users/0ef9a1f304184d5cbc95d7d69ef447d7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:58.183681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef9a1f304184d5cbc95d7d69ef447d7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:58.186457 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:58.414070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef9a1f304184d5cbc95d7d69ef447d7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:58.414803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef9a1f304184d5cbc95d7d69ef447d7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:58.422703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef9a1f304184d5cbc95d7d69ef447d7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:58.434914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef9a1f304184d5cbc95d7d69ef447d7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:58.512697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef9a1f304184d5cbc95d7d69ef447d7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:58.520397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1028/2035] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:58 2022] DELETE /identity/v3/users/0ef9a1f304184d5cbc95d7d69ef447d7 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:58.525028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:58.525526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:58.525623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:58.546764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:36:58.547982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:58.614860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1008/2036] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:36:58.625369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:58.626854 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:58.650354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:58.690210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319a2fe-ec0b-4309-adf9-42b45ba9162c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:58.693651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:58.694183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:58.694467 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:58.694747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:58.705230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:58.706646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:58.717891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1029/2037] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:36:58.776025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:59.303494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:59.305208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:59.318597 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41ce6b-73f6-4e60-8a4b-dcdd7bf01c4d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:59.323728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:59.324385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:59.324700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:59.324977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] PATH_INFO: `/v3/projects/48133a3d94214dff8aa2088e11d259d2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:59.326202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] RBAC: Authorizing `identity:delete_project(project_id=48133a3d94214dff8aa2088e11d259d2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:59.328172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:59.336147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:59.472676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:59.482268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48133a3d94214dff8aa2088e11d259d2', 'request_id': 'req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:59.598462 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48133a3d94214dff8aa2088e11d259d2', 'request_id': 'req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:59.605747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48133a3d94214dff8aa2088e11d259d2', 'request_id': 'req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:59.618465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] The token cache is being invalidate because project 48133a3d94214dff8aa2088e11d259d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:36:59.618834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:59.619511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:59.621118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1009/2038] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:36:59 2022] DELETE /identity/v3/projects/48133a3d94214dff8aa2088e11d259d2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:36:59.625852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:59.635591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5d6a4-0eae-43b9-aac3-64efaeb5dc0b service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:59.701482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:59.702245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:59.702539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:59.702836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] PATH_INFO: `/v3/users/4bfb8d76e1294f6dbd966c2c6a8d402b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:59.704311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bfb8d76e1294f6dbd966c2c6a8d402b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:59.706454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:59.722254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:36:59.722254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:36:59.760928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbdb3-c0d7-44ac-a87b-b8caffa8177b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:36:59.771229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:36:59.771229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:36:59.771229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:36:59.771229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] PATH_INFO: `/v3/users/3fff6a0c1fe84662871a55e147960bd6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:36:59.771580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fff6a0c1fe84662871a55e147960bd6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:36:59.771580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:36:59.895098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:36:59.985592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bfb8d76e1294f6dbd966c2c6a8d402b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:59.986290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bfb8d76e1294f6dbd966c2c6a8d402b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:36:59.992518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fff6a0c1fe84662871a55e147960bd6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.000266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bfb8d76e1294f6dbd966c2c6a8d402b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.001907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fff6a0c1fe84662871a55e147960bd6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.004832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fff6a0c1fe84662871a55e147960bd6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.017372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bfb8d76e1294f6dbd966c2c6a8d402b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.112239 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fff6a0c1fe84662871a55e147960bd6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.112897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff692735-a9e1-4af9-8e08-1c67dcdab115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fff6a0c1fe84662871a55e147960bd6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.131295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1010/2039] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:59 2022] DELETE /identity/v3/users/3fff6a0c1fe84662871a55e147960bd6 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:37:00.135709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:00.135999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:00.136284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:00.155713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:00.156979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:00.186536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bfb8d76e1294f6dbd966c2c6a8d402b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.193735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1030/2040] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:36:59 2022] DELETE /identity/v3/users/4bfb8d76e1294f6dbd966c2c6a8d402b => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:00.220182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1011/2041] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:00.230169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:00.231769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:00.294116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7897112-2c0d-4aba-9e6e-8bbc9dda7981 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:00.299121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:00.299751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:00.300032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:00.300306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:00.364158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:00.365641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:00.375388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1031/2042] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:37:00.404435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:00.679661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:00.681144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:00.684220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:00.699677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41158726-eb39-4694-90c1-5aa7f3b4dc73 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:00.705433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:00.706208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:00.706556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:00.706902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] PATH_INFO: `/v3/projects/be2bb3de9789417aaa0337b128ceda9c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:00.708330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2bb3de9789417aaa0337b128ceda9c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:00.710552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:00.829550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2bb3de9789417aaa0337b128ceda9c', 'request_id': 'req-532a1398-d82d-493a-a1f5-a6f92ee5b510'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.851320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:00.852957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:00.866733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deeb80-2c87-49e8-8988-d5387961b17a service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:00.905751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2bb3de9789417aaa0337b128ceda9c', 'request_id': 'req-532a1398-d82d-493a-a1f5-a6f92ee5b510'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.907017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2bb3de9789417aaa0337b128ceda9c', 'request_id': 'req-532a1398-d82d-493a-a1f5-a6f92ee5b510'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.919174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] The token cache is being invalidate because project be2bb3de9789417aaa0337b128ceda9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:00.919628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.920350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:00.922068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1012/2043] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:00 2022] DELETE /identity/v3/projects/be2bb3de9789417aaa0337b128ceda9c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:00.926195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:00.928661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:00.928661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:00.928661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:00.928661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] PATH_INFO: `/v3/projects/5f908335d06a4da4922bbdde9b8cc1ce` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:00.928999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:00.929098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f908335d06a4da4922bbdde9b8cc1ce)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:00.930574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:00.941453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a1398-d82d-493a-a1f5-a6f92ee5b510 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:00.946494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:00.947148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:00.947435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:00.947709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] PATH_INFO: `/v3/users/fd15ebc747e3455c85a2ee803c6735d6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:00.948925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd15ebc747e3455c85a2ee803c6735d6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:00.951452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:01.081290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f908335d06a4da4922bbdde9b8cc1ce', 'request_id': 'req-4fdf6472-a6cf-4249-869a-8d69a0b39107'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.088543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f908335d06a4da4922bbdde9b8cc1ce', 'request_id': 'req-4fdf6472-a6cf-4249-869a-8d69a0b39107'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.094413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f908335d06a4da4922bbdde9b8cc1ce', 'request_id': 'req-4fdf6472-a6cf-4249-869a-8d69a0b39107'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.192988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] The token cache is being invalidate because project 5f908335d06a4da4922bbdde9b8cc1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:01.193245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.193856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.195402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1032/2044] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:00 2022] DELETE /identity/v3/projects/5f908335d06a4da4922bbdde9b8cc1ce => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:37:01.199663 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:01.202443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:01.207744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd15ebc747e3455c85a2ee803c6735d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.215046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd15ebc747e3455c85a2ee803c6735d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.215799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd15ebc747e3455c85a2ee803c6735d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.272114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf6472-a6cf-4249-869a-8d69a0b39107 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:01.277575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:01.278229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:01.278513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:01.279429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] PATH_INFO: `/v3/users/7dc228f53dd04005973c1f694c224633` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:01.280923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc228f53dd04005973c1f694c224633)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:01.282901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:01.283585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd15ebc747e3455c85a2ee803c6735d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.284220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd15ebc747e3455c85a2ee803c6735d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.292181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:01.305565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1013/2045] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:37:00 2022] DELETE /identity/v3/users/fd15ebc747e3455c85a2ee803c6735d6 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:37:01.491763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc228f53dd04005973c1f694c224633'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.492729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc228f53dd04005973c1f694c224633'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.500721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc228f53dd04005973c1f694c224633'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.512736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc228f53dd04005973c1f694c224633'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.615486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:01.631470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-67cacb7d-e6c3-4606-beb2-42e62f84e7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc228f53dd04005973c1f694c224633'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:01.638834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1033/2046] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:37:01 2022] DELETE /identity/v3/users/7dc228f53dd04005973c1f694c224633 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:01.741892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:01.744099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:01.828778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6f68d-15eb-4052-818b-007a06f51609 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:01.887545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:01.888074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:01.889042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:01.899438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] PATH_INFO: `/v3/projects/a935c1c8a2fe4509ac717df4bdea4865` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:01.899438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a935c1c8a2fe4509ac717df4bdea4865)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:01.899438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:01.988414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a935c1c8a2fe4509ac717df4bdea4865', 'request_id': 'req-7472558d-6ef0-4ef0-ade7-eb674202095a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.071133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a935c1c8a2fe4509ac717df4bdea4865', 'request_id': 'req-7472558d-6ef0-4ef0-ade7-eb674202095a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.078263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a935c1c8a2fe4509ac717df4bdea4865', 'request_id': 'req-7472558d-6ef0-4ef0-ade7-eb674202095a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.089770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] The token cache is being invalidate because project a935c1c8a2fe4509ac717df4bdea4865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:02.090066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.090749 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.092489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1014/2047] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:01 2022] DELETE /identity/v3/projects/a935c1c8a2fe4509ac717df4bdea4865 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:02.098554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b38bc-61b2-497e-a6c0-a2bca5aa6f41 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.098848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b38bc-61b2-497e-a6c0-a2bca5aa6f41 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.099125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b38bc-61b2-497e-a6c0-a2bca5aa6f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.120162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-684b38bc-61b2-497e-a6c0-a2bca5aa6f41 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:02.121422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-684b38bc-61b2-497e-a6c0-a2bca5aa6f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.170931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.172643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.195005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1034/2048] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:02.196910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e704f2-4b92-4491-85c8-5ff3685dee53 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.197204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e704f2-4b92-4491-85c8-5ff3685dee53 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.197486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e704f2-4b92-4491-85c8-5ff3685dee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.217881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d0e704f2-4b92-4491-85c8-5ff3685dee53 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:02.219287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d0e704f2-4b92-4491-85c8-5ff3685dee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.227599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1035/2049] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:02.231593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e704f2-4b92-4491-85c8-5ff3685dee53 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.241981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e704f2-4b92-4491-85c8-5ff3685dee53 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.246480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.247096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.247420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.247768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.249027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:37:02.249332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.250950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.254152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472558d-6ef0-4ef0-ade7-eb674202095a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.258919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.260117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.260402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.260796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] PATH_INFO: `/v3/projects/798c9d548f784a8d9c475bf8f96918c9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.261888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=798c9d548f784a8d9c475bf8f96918c9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.263026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1036/2050] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:37:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:37:02.264560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.268085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.278128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b45c-a8c4-4592-87da-1da03ef14213 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.283564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.284201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.284512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.284791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.285459 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.287452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.393595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798c9d548f784a8d9c475bf8f96918c9', 'request_id': 'req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.414955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1037/2051] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/projects => generated 384 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:37:02.416042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.482806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c7165-9ddb-4bda-a30b-c58ab5da19cc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.487641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.488273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.488385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798c9d548f784a8d9c475bf8f96918c9', 'request_id': 'req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.488667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.488949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.489626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.491608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.494720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798c9d548f784a8d9c475bf8f96918c9', 'request_id': 'req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.506819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] The token cache is being invalidate because project 798c9d548f784a8d9c475bf8f96918c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:02.507110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.511687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d1a90bf7-e45a-4ef8-bf67-e2b160b2cbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:02.511687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1015/2052] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:02 2022] DELETE /identity/v3/projects/798c9d548f784a8d9c475bf8f96918c9 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:02.515117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268270c2-c551-4811-9c9a-19136b1b5e94 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.515406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268270c2-c551-4811-9c9a-19136b1b5e94 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.515684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268270c2-c551-4811-9c9a-19136b1b5e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.540804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-268270c2-c551-4811-9c9a-19136b1b5e94 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:02.542005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-268270c2-c551-4811-9c9a-19136b1b5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.551087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1016/2053] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:02.558834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a49898-7d41-4e0b-a84f-e337d8212060 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.559123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a49898-7d41-4e0b-a84f-e337d8212060 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.559405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a49898-7d41-4e0b-a84f-e337d8212060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.574794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:02.580812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-18a49898-7d41-4e0b-a84f-e337d8212060 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:02.582011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-18a49898-7d41-4e0b-a84f-e337d8212060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.589803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1017/2054] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:02.594677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a49898-7d41-4e0b-a84f-e337d8212060 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.604506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a49898-7d41-4e0b-a84f-e337d8212060 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.610714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.611373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.611651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.611937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.612106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1038/2055] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:37:02.612732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:37:02.612971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.614577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.616044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.617937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.630473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1018/2056] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:37:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:37:02.631444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b8cf2-1710-4136-b604-e95b9d6e3e8f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.633027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.636816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.637445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.637730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.638010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.638724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.640524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.643270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c319-936c-4717-a12a-d07e35cabaea admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.648013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.648740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.649011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.649291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.649960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.650793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1039/2057] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:02 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:37:02.652098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.655353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.666401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1caf3b-4614-480e-b2c4-fa38776c2018 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.672273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.672950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.673227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.673511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.675027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.675817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.686986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1040/2058] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:02 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:37:02.691292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.702524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434542d-574d-40fe-b596-0677a4d5caab admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.707285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d7c64-8fe2-4b01-bbd8-d888a591ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.707940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7c64-8fe2-4b01-bbd8-d888a591ff24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.708228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7c64-8fe2-4b01-bbd8-d888a591ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.708546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7c64-8fe2-4b01-bbd8-d888a591ff24 admin admin] PATH_INFO: `/v3/projects/b507debe3c694bb88155992d7a741c02/users/8161934c978e41a5a928904b0bc994e4/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.724053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d7c64-8fe2-4b01-bbd8-d888a591ff24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b507debe3c694bb88155992d7a741c02, user_id=8161934c978e41a5a928904b0bc994e4, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.726745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d7c64-8fe2-4b01-bbd8-d888a591ff24 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.770451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1019/2059] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/projects => generated 368 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:37:02.774865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.798862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1041/2060] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:37:02 2022] PUT /identity/v3/projects/b507debe3c694bb88155992d7a741c02/users/8161934c978e41a5a928904b0bc994e4/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:37:02.811926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.812216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.812523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.839149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba214a-8f8a-4025-b54d-961edf0de7c5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.842713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] MFA Rules not processed for user `8161934c978e41a5a928904b0bc994e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:02.844081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.899439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.900119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.900407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.900728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:02.901403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:02.903323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:02.914422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1042/2061] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:02.926829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:02.938084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:02.958420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:02.988701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cc130-1742-46d4-b53f-52c007407f26 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:02.992367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:02.993009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:02.993296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:02.993577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.002804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:03.004253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:03.013411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1043/2062] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:37:03.033289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:03.045997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1020/2063] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:37:02 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:37:03.046282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fb659-36af-42fd-8de1-9c87fecbf64f service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:03.049468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:03.050114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.050384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.050690 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] PATH_INFO: `/v3/users/8161934c978e41a5a928904b0bc994e4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.051348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:03.051888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8161934c978e41a5a928904b0bc994e4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:03.053905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:03.061507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f5fe0-f1c2-4719-8d54-c376118646d9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:03.066712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:03.067349 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.071475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.071475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.071475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:03.071475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:03.081075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1021/2064] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:03 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:37:03.085241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:03.096475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc8efb-5f63-465d-a6e8-d715f3677213 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:03.101262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:03.101905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.102203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.102487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.103329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:03.105092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:03.117112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1022/2065] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:03 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:37:03.120954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:03.131308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f637-23d5-43a5-b058-5b47ca70039b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:03.135845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b9713-ba9b-487a-8261-d9764ee656b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:03.136532 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b9713-ba9b-487a-8261-d9764ee656b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.136825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b9713-ba9b-487a-8261-d9764ee656b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.137111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b9713-ba9b-487a-8261-d9764ee656b6 admin admin] PATH_INFO: `/v3/projects/115c7877ef274b759c5de1b5fd4b5267/users/9128bdfb7a09410ab22fd64cd27819b2/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.152178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b9713-ba9b-487a-8261-d9764ee656b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115c7877ef274b759c5de1b5fd4b5267, user_id=9128bdfb7a09410ab22fd64cd27819b2, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:03.155628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b9713-ba9b-487a-8261-d9764ee656b6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:03.306905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1023/2066] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:37:03 2022] PUT /identity/v3/projects/115c7877ef274b759c5de1b5fd4b5267/users/9128bdfb7a09410ab22fd64cd27819b2/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:37:03.318651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.318941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.319246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.342725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8161934c978e41a5a928904b0bc994e4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:03.343434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8161934c978e41a5a928904b0bc994e4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:03.350519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8161934c978e41a5a928904b0bc994e4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:03.351478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] MFA Rules not processed for user `9128bdfb7a09410ab22fd64cd27819b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:03.352787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:03.363478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8161934c978e41a5a928904b0bc994e4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:03.418118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1024/2067] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:03.430602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:03.431924 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:03.468197 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-895a9069-5aff-4899-a407-2522c2d16159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8161934c978e41a5a928904b0bc994e4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:03.480965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1044/2068] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:37:03 2022] DELETE /identity/v3/users/8161934c978e41a5a928904b0bc994e4 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:37:03.482163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.482558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.482926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.495179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dca0bd-3dad-4b25-a9f8-ec406909a6f2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:03.505036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:03.506363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:03.550273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:03.551023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.551264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.551545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.553576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:03.585857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1045/2069] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:03.585857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:03.585857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:03.623223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:03.625411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:03.635531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1025/2070] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:37:03.648439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de7732-609c-4553-9f8e-04b7c5e7b11f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:03.652684 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa77845-44c2-4ffd-870d-cb3655caadc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:03.653317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa77845-44c2-4ffd-870d-cb3655caadc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:03.653752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa77845-44c2-4ffd-870d-cb3655caadc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:03.654053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa77845-44c2-4ffd-870d-cb3655caadc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:03.668573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa77845-44c2-4ffd-870d-cb3655caadc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:03.671458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa77845-44c2-4ffd-870d-cb3655caadc3 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:03.681337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1046/2071] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:37:04.234923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:04.236996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:04.254113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bdf65-0eaf-46f4-8c23-c5b611553705 service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:04.259657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:04.260689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:04.261074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:04.261452 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] PATH_INFO: `/v3/projects/b507debe3c694bb88155992d7a741c02` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:04.262897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b507debe3c694bb88155992d7a741c02)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:04.268241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:04.370377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b507debe3c694bb88155992d7a741c02', 'request_id': 'req-f91dabed-616b-4cc4-a424-73bd48403ddb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:04.459970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b507debe3c694bb88155992d7a741c02', 'request_id': 'req-f91dabed-616b-4cc4-a424-73bd48403ddb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:04.470023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b507debe3c694bb88155992d7a741c02', 'request_id': 'req-f91dabed-616b-4cc4-a424-73bd48403ddb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:04.482360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] The token cache is being invalidate because project b507debe3c694bb88155992d7a741c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:04.482665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:04.483354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f91dabed-616b-4cc4-a424-73bd48403ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:04.485737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1026/2072] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:04 2022] DELETE /identity/v3/projects/b507debe3c694bb88155992d7a741c02 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:04.490847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f94d6-5148-4dc8-9602-03b419d97b3b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:04.492192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:04.496736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f94d6-5148-4dc8-9602-03b419d97b3b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:04.497038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f94d6-5148-4dc8-9602-03b419d97b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:04.517169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e74f94d6-5148-4dc8-9602-03b419d97b3b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:04.520332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e74f94d6-5148-4dc8-9602-03b419d97b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:04.580621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1047/2073] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:04.588191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a0f7-a3dd-42f2-8e71-690229abadf9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:04.588599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a0f7-a3dd-42f2-8e71-690229abadf9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:04.588938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a0f7-a3dd-42f2-8e71-690229abadf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:04.600643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:04.626544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-29e3a0f7-a3dd-42f2-8e71-690229abadf9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:04.628213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-29e3a0f7-a3dd-42f2-8e71-690229abadf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:04.637291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1027/2074] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:04.643596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f94d6-5148-4dc8-9602-03b419d97b3b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:04.653214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f94d6-5148-4dc8-9602-03b419d97b3b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:04.658470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:04.659135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:04.659393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:04.659678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:04.660391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:37:04.660734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:04.662350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:04.675373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1048/2075] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:37:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:37:04.680274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a0f7-a3dd-42f2-8e71-690229abadf9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:04.697053 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a0f7-a3dd-42f2-8e71-690229abadf9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:04.701343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:04.702173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:04.702640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:04.703106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:04.703873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:04.707311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:04.817583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:04.818296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1028/2076] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:37:04 2022] POST /identity/v3/projects => generated 376 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:37:04.833315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b04a-1bc8-4eb6-b01d-e2e597c564c6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:04.907544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:04.908170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:04.908551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:04.908911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:04.909804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:04.911744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:05.020031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1049/2077] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:37:04 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:37:05.024790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:05.037534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07f21d-36e1-4849-8e27-563c694fba13 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:05.044602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:05.046348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:05.046813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:05.047264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:05.048433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:05.051250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:05.063999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1029/2078] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:37:05.068143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:05.079323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b5414-605c-4dad-a670-b994c8fecc46 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:05.084036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c64b27-44df-4008-9336-e38fb5fb0936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:05.084680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c64b27-44df-4008-9336-e38fb5fb0936 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:05.084961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c64b27-44df-4008-9336-e38fb5fb0936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:05.085237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c64b27-44df-4008-9336-e38fb5fb0936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:05.085932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c64b27-44df-4008-9336-e38fb5fb0936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:05.087522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c64b27-44df-4008-9336-e38fb5fb0936 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:05.099411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1050/2079] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:37:05.106995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:05.124896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e44ab9-44a1-4e03-9a9d-23c48bb31820 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:05.132702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:05.134399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:05.134825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:05.135531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] PATH_INFO: `/v3/projects/1dcf34beea014060b1257244e6aa5226/users/6b5e9d7749e040998c3ef03a8408897c/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:05.160649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dcf34beea014060b1257244e6aa5226, user_id=6b5e9d7749e040998c3ef03a8408897c, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:05.163480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:05.216331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1030/2080] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:37:05 2022] PUT /identity/v3/projects/1dcf34beea014060b1257244e6aa5226/users/6b5e9d7749e040998c3ef03a8408897c/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:05.227509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:05.228063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:05.228559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:05.265813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] MFA Rules not processed for user `6b5e9d7749e040998c3ef03a8408897c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:05.267340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:05.339698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1051/2081] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:05.353131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:05.355001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:05.414787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bd1b-898d-4912-906c-ab30b2085d90 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:05.419297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345ee7b-1d1a-4c82-8399-8ddd9c4bb237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:05.420195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ee7b-1d1a-4c82-8399-8ddd9c4bb237 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:05.420647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ee7b-1d1a-4c82-8399-8ddd9c4bb237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:05.420914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ee7b-1d1a-4c82-8399-8ddd9c4bb237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:05.430207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345ee7b-1d1a-4c82-8399-8ddd9c4bb237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:05.431760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345ee7b-1d1a-4c82-8399-8ddd9c4bb237 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:05.442738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1031/2082] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:05.452700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:05.506728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:05.786132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:05.871870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:06.131328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:06.386627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:08.363617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:08.365120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:08.435164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3d377-934a-4aa5-8228-d178bee6dc0c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:08.440652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:08.441424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:08.441683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:08.441968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] PATH_INFO: `/v3/users/113f316bcbb9422283b4ccdc86a56e56` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:08.443184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=113f316bcbb9422283b4ccdc86a56e56)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:08.445184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:08.651188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113f316bcbb9422283b4ccdc86a56e56'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:08.651772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113f316bcbb9422283b4ccdc86a56e56'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:08.660131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113f316bcbb9422283b4ccdc86a56e56'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:08.667868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:08.673528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113f316bcbb9422283b4ccdc86a56e56'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:08.745549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113f316bcbb9422283b4ccdc86a56e56'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:08.752615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1052/2083] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:37:08 2022] DELETE /identity/v3/users/113f316bcbb9422283b4ccdc86a56e56 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:08.757323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:08.757838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:08.757970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:08.777035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:08.778481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:08.842105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1032/2084] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:08.852344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:08.854015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:08.914476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f3811-1acd-43af-a484-23bdbeb79c61 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:08.918005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2081ee-58f4-4cef-9ac5-3c0a7a82dbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:08.918628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2081ee-58f4-4cef-9ac5-3c0a7a82dbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:08.918911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2081ee-58f4-4cef-9ac5-3c0a7a82dbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:08.919191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2081ee-58f4-4cef-9ac5-3c0a7a82dbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:08.930216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2081ee-58f4-4cef-9ac5-3c0a7a82dbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:08.932781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2081ee-58f4-4cef-9ac5-3c0a7a82dbc9 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:08.943384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1053/2085] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:09.352222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:09.353724 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:09.366818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7393cc-67a7-403a-8aee-23c7166ade96 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:09.371819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:09.372585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:09.373344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:09.373667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] PATH_INFO: `/v3/projects/915fadab15dc4f249412d83fdfab6f6d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:09.375042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=915fadab15dc4f249412d83fdfab6f6d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:09.376994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:09.488750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '915fadab15dc4f249412d83fdfab6f6d', 'request_id': 'req-c5dfeacc-2710-48fb-aa5e-3efd5a510220'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:09.556991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '915fadab15dc4f249412d83fdfab6f6d', 'request_id': 'req-c5dfeacc-2710-48fb-aa5e-3efd5a510220'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:09.566853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '915fadab15dc4f249412d83fdfab6f6d', 'request_id': 'req-c5dfeacc-2710-48fb-aa5e-3efd5a510220'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:09.583785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] The token cache is being invalidate because project 915fadab15dc4f249412d83fdfab6f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:09.584085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:09.584802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5dfeacc-2710-48fb-aa5e-3efd5a510220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:09.586412 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1033/2086] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:09 2022] DELETE /identity/v3/projects/915fadab15dc4f249412d83fdfab6f6d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:09.592150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c055bc-18d9-49a3-bf30-3cd759a9afc2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:09.592783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c055bc-18d9-49a3-bf30-3cd759a9afc2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:09.592783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c055bc-18d9-49a3-bf30-3cd759a9afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:09.628985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f6c055bc-18d9-49a3-bf30-3cd759a9afc2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:09.630301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f6c055bc-18d9-49a3-bf30-3cd759a9afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:09.712810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1054/2087] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:09.721050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33620312-9ed7-42cd-8196-2ee591c4aa5d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:09.721340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33620312-9ed7-42cd-8196-2ee591c4aa5d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:09.721619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33620312-9ed7-42cd-8196-2ee591c4aa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:09.744329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-33620312-9ed7-42cd-8196-2ee591c4aa5d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:09.746308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-33620312-9ed7-42cd-8196-2ee591c4aa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:09.757111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1034/2088] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:09.761691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c055bc-18d9-49a3-bf30-3cd759a9afc2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:09.773522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c055bc-18d9-49a3-bf30-3cd759a9afc2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:09.778176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:09.779376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:09.779770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:09.780055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:09.780993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:37:09.781313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:09.782945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:09.799791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1055/2089] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:37:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:37:09.805084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33620312-9ed7-42cd-8196-2ee591c4aa5d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:09.812030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:09.817539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33620312-9ed7-42cd-8196-2ee591c4aa5d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:09.823509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:09.824191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:09.824511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:09.825063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:09.826727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:09.829052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:09.945085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1035/2090] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:37:09 2022] POST /identity/v3/projects => generated 372 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:37:09.950581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:09.966063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0533-4656-4c1a-af06-25a20ed8c44c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:10.053039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:10.054077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:10.054591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:10.055105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:10.056279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:10.059270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:10.169330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1056/2091] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:37:09 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:37:10.176921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:10.190783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf5ef6-7f1b-42e9-8157-b8a754c8a4e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:10.198138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:10.198772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:10.199064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:10.199340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:10.200182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:10.202467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:10.216653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1036/2092] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:10 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:37:10.227526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:10.236507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:10.241971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4534ba-bd44-4ee6-a261-697b7c001327 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:10.250454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77098191-2603-49ce-8a94-07d4c2bd2232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:10.251405 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77098191-2603-49ce-8a94-07d4c2bd2232 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:10.251876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77098191-2603-49ce-8a94-07d4c2bd2232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:10.252319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77098191-2603-49ce-8a94-07d4c2bd2232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:10.253297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77098191-2603-49ce-8a94-07d4c2bd2232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:10.255063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77098191-2603-49ce-8a94-07d4c2bd2232 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:10.270501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1057/2093] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:10 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:37:10.276440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:10.289394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9e0d3-7c44-4fb4-a1fb-404309505c0c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:10.295314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:10.296126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:10.296657 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:10.297134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] PATH_INFO: `/v3/projects/eb5cc7e509e84773bec86203332e38af/users/8c8e0d314c074eaa91f6b16c6777cf1e/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:10.319358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5cc7e509e84773bec86203332e38af, user_id=8c8e0d314c074eaa91f6b16c6777cf1e, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:10.323636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:10.432648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1037/2094] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:37:10 2022] PUT /identity/v3/projects/eb5cc7e509e84773bec86203332e38af/users/8c8e0d314c074eaa91f6b16c6777cf1e/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:10.439153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924c417-3adf-489e-a98f-4a416f5d20bc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:10.439444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924c417-3adf-489e-a98f-4a416f5d20bc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:10.439725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924c417-3adf-489e-a98f-4a416f5d20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:10.460646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9924c417-3adf-489e-a98f-4a416f5d20bc None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:10.461872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9924c417-3adf-489e-a98f-4a416f5d20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:10.521444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1058/2095] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:10.532052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:10.533876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:10.603305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6c0c9-16f0-4613-9507-4baeee9fcff8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:10.609647 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:10.610281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:10.610558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:10.610837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:10.620918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:10.622361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:10.632516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1038/2096] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:12.291589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632a0ce-8768-40c0-bd3d-b976ea8df780 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:12.292134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632a0ce-8768-40c0-bd3d-b976ea8df780 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:12.292134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632a0ce-8768-40c0-bd3d-b976ea8df780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:12.325100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3632a0ce-8768-40c0-bd3d-b976ea8df780 None None] MFA Rules not processed for user `8c8e0d314c074eaa91f6b16c6777cf1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:12.326807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3632a0ce-8768-40c0-bd3d-b976ea8df780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:12.402029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1059/2097] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:12.573941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:12.576268 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:12.635897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad74fc1-501c-4f16-aac8-205b3fe32c07 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:12.640408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:12.641060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:12.641343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:12.641616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:12.650013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:12.651437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:12.661854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1039/2098] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:20.749841 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab29a2-1b5d-40b5-857c-a598bb1b79ce None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:20.750381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab29a2-1b5d-40b5-857c-a598bb1b79ce None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:20.750381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab29a2-1b5d-40b5-857c-a598bb1b79ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:20.770694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d1ab29a2-1b5d-40b5-857c-a598bb1b79ce None None] MFA Rules not processed for user `8c8e0d314c074eaa91f6b16c6777cf1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:20.772277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d1ab29a2-1b5d-40b5-857c-a598bb1b79ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:20.782792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1060/2099] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:22.657870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=74,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:30.763820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=65 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:36.441362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:36.444506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:36.478866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae71c1-7134-4280-8d57-51a6b517f0bb service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:36.484746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:36.485685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:36.488006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:36.488448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] PATH_INFO: `/v3/users/6b5e9d7749e040998c3ef03a8408897c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:36.489796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5e9d7749e040998c3ef03a8408897c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:36.492658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:36.501644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e6f34-6547-4451-b161-aed7a2d6681e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:36.502209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e6f34-6547-4451-b161-aed7a2d6681e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:36.502296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e6f34-6547-4451-b161-aed7a2d6681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:36.531054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f79e6f34-6547-4451-b161-aed7a2d6681e None None] MFA Rules not processed for user `c2bf7ea384a142d1bc23b23ab503fca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:36.532414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f79e6f34-6547-4451-b161-aed7a2d6681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:36.593225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1061/2100] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:36.657358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5e9d7749e040998c3ef03a8408897c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:36.665869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5e9d7749e040998c3ef03a8408897c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:36.666720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5e9d7749e040998c3ef03a8408897c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:36.733275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5e9d7749e040998c3ef03a8408897c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:36.733901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5e9d7749e040998c3ef03a8408897c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:36.765692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1040/2101] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:37:36 2022] DELETE /identity/v3/users/6b5e9d7749e040998c3ef03a8408897c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:36.770196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:36.771017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:36.771641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:36.805871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:36.807466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:36.897851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1062/2102] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:36.914179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:36.915636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:37.022365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c5212-4076-4022-8059-9e40caac09a4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:37.028155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93a78-8c5e-40b4-b1b6-77e88f49c004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:37.030210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93a78-8c5e-40b4-b1b6-77e88f49c004 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:37.030505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93a78-8c5e-40b4-b1b6-77e88f49c004 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:37.030788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93a78-8c5e-40b4-b1b6-77e88f49c004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:37.046840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93a78-8c5e-40b4-b1b6-77e88f49c004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:37.049250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93a78-8c5e-40b4-b1b6-77e88f49c004 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:37.061231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1041/2103] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:37.575577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:37.578093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:37.613569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60288b68-d285-482a-9e6e-dbd4ecfb56b0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:37.625942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:37.627716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:37.627716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:37.627716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] PATH_INFO: `/v3/projects/1dcf34beea014060b1257244e6aa5226` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:37.630063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dcf34beea014060b1257244e6aa5226)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:37.634195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:37.758271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dcf34beea014060b1257244e6aa5226', 'request_id': 'req-364bf2de-215b-46e7-8c5e-e91618c3a20e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:37.767218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dcf34beea014060b1257244e6aa5226', 'request_id': 'req-364bf2de-215b-46e7-8c5e-e91618c3a20e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:37.774849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dcf34beea014060b1257244e6aa5226', 'request_id': 'req-364bf2de-215b-46e7-8c5e-e91618c3a20e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:37.852778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] The token cache is being invalidate because project 1dcf34beea014060b1257244e6aa5226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:37.853293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:37.854074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-364bf2de-215b-46e7-8c5e-e91618c3a20e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:37.855972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1063/2104] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:37 2022] DELETE /identity/v3/projects/1dcf34beea014060b1257244e6aa5226 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:37.863862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641a2d5-920e-4b6b-9f06-f37af3736670 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:37.864673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641a2d5-920e-4b6b-9f06-f37af3736670 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:37.864813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641a2d5-920e-4b6b-9f06-f37af3736670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:37.887450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8641a2d5-920e-4b6b-9f06-f37af3736670 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:37.888723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8641a2d5-920e-4b6b-9f06-f37af3736670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:37.956610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1042/2105] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:37.964794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bbda-3b57-4c4e-8bd6-62a219911129 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:37.964980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bbda-3b57-4c4e-8bd6-62a219911129 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:37.965260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bbda-3b57-4c4e-8bd6-62a219911129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:37.989037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-39d7bbda-3b57-4c4e-8bd6-62a219911129 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:37.990767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39d7bbda-3b57-4c4e-8bd6-62a219911129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:38.004915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1064/2106] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:38.011706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641a2d5-920e-4b6b-9f06-f37af3736670 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:38.025277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641a2d5-920e-4b6b-9f06-f37af3736670 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:38.034188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:38.038448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.038448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.038448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.038448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:37:38.038448 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:38.040792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:38.064324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1043/2107] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:37:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:37:38.071278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7bbda-3b57-4c4e-8bd6-62a219911129 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:38.090596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7bbda-3b57-4c4e-8bd6-62a219911129 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:38.097426 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:38.098194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.098496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.098775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.099463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:38.101501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:38.215697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:38.224526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1065/2108] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:37:38 2022] POST /identity/v3/projects => generated 376 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:37:38.229987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:38.238890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:38.244593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bb0aa-e4da-49b5-bf43-9a5706e75507 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:38.317887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:38.318511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.318793 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.319334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.320055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:38.322016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:38.428292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1044/2109] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:37:38 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:37:38.432413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:38.442879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c93151-aaca-41d3-9480-188b95da7031 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:38.448951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:38.449613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.449924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.450177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.450900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:38.453108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:38.465225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1066/2110] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:38 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:37:38.470436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:38.480676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a96825-633b-4a30-9b3b-5b10b4c92887 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:38.487320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73f8b-0290-4488-9c4a-840e02ebf8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:38.487946 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73f8b-0290-4488-9c4a-840e02ebf8ba admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.488232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73f8b-0290-4488-9c4a-840e02ebf8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.488548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73f8b-0290-4488-9c4a-840e02ebf8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.489343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73f8b-0290-4488-9c4a-840e02ebf8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:38.490943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73f8b-0290-4488-9c4a-840e02ebf8ba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:38.503403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1045/2111] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:38 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:37:38.508448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:38.519804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade82d8-b45c-4de1-a83b-81b4d0e9073d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:38.524815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:38.526511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.527031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.527326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] PATH_INFO: `/v3/projects/a387bd3b1856478f8b31ed2830d75229/users/7ea459868d0445c8b4032d77072fd248/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.546506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a387bd3b1856478f8b31ed2830d75229, user_id=7ea459868d0445c8b4032d77072fd248, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:38.550264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:38.624971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1067/2112] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:37:38 2022] PUT /identity/v3/projects/a387bd3b1856478f8b31ed2830d75229/users/7ea459868d0445c8b4032d77072fd248/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:38.636676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5aaa7-b259-4c2b-86be-bbe9ecd61ef6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.636970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5aaa7-b259-4c2b-86be-bbe9ecd61ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.637297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5aaa7-b259-4c2b-86be-bbe9ecd61ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.671714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2eb5aaa7-b259-4c2b-86be-bbe9ecd61ef6 None None] MFA Rules not processed for user `7ea459868d0445c8b4032d77072fd248`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:38.672995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2eb5aaa7-b259-4c2b-86be-bbe9ecd61ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:38.752857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1046/2113] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:38.764936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:38.766738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:38.837572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e948bcc-b4f6-4a2e-a2c5-8f955446fe32 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:38.843081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc45838-22b3-4dab-a1e8-ebe8f0aa8f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:38.845608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45838-22b3-4dab-a1e8-ebe8f0aa8f6c service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:38.845608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45838-22b3-4dab-a1e8-ebe8f0aa8f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:38.845608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45838-22b3-4dab-a1e8-ebe8f0aa8f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:38.854887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc45838-22b3-4dab-a1e8-ebe8f0aa8f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:38.856886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc45838-22b3-4dab-a1e8-ebe8f0aa8f6c service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:38.868512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1068/2114] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:43.280552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5aaa7-b259-4c2b-86be-bbe9ecd61ef6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:43.353319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5aaa7-b259-4c2b-86be-bbe9ecd61ef6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:43.359281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:43.360005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:43.360294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:43.360606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] PATH_INFO: `/v3/users/7ea459868d0445c8b4032d77072fd248` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:43.361824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea459868d0445c8b4032d77072fd248)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:43.363762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:43.377689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:43.612515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea459868d0445c8b4032d77072fd248'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:43.619893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea459868d0445c8b4032d77072fd248'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:43.620696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea459868d0445c8b4032d77072fd248'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:43.749772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea459868d0445c8b4032d77072fd248'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:43.750427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea459868d0445c8b4032d77072fd248'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:43.772562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1047/2115] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:37:43 2022] DELETE /identity/v3/users/7ea459868d0445c8b4032d77072fd248 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:43.777199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:43.777704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:43.777798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:43.797489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:43.798710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:43.880935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1069/2116] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:43.891215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:43.892812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:43.960271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f82e9-3bc0-49b7-b001-e29fdf7e4cb1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:43.964335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c88b2-5141-4405-b387-94f5702049dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:43.965095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c88b2-5141-4405-b387-94f5702049dc service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:43.965380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c88b2-5141-4405-b387-94f5702049dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:43.965659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c88b2-5141-4405-b387-94f5702049dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:43.977671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c88b2-5141-4405-b387-94f5702049dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:43.979546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c88b2-5141-4405-b387-94f5702049dc service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:44.001758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1048/2117] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:45.253337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:45.254852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:45.269729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1b138-1b9d-40d6-9f48-2fc1fba9e643 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:45.275037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:45.278188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:45.278188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:45.278188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] PATH_INFO: `/v3/projects/a387bd3b1856478f8b31ed2830d75229` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:45.278188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a387bd3b1856478f8b31ed2830d75229)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:45.279530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:45.509644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a387bd3b1856478f8b31ed2830d75229', 'request_id': 'req-cf74b465-e1fa-4f7f-bb24-52d441060d21'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:45.521968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a387bd3b1856478f8b31ed2830d75229', 'request_id': 'req-cf74b465-e1fa-4f7f-bb24-52d441060d21'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:45.548417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a387bd3b1856478f8b31ed2830d75229', 'request_id': 'req-cf74b465-e1fa-4f7f-bb24-52d441060d21'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:45.551063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:45.658800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] The token cache is being invalidate because project a387bd3b1856478f8b31ed2830d75229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:45.659106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:45.660664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cf74b465-e1fa-4f7f-bb24-52d441060d21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:45.666126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1070/2118] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:45 2022] DELETE /identity/v3/projects/a387bd3b1856478f8b31ed2830d75229 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:45.671700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d271e-764a-499a-8d6f-c782bd1186a0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:45.672893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d271e-764a-499a-8d6f-c782bd1186a0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:45.673380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d271e-764a-499a-8d6f-c782bd1186a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:45.703936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6c16-24ba-45f8-8947-01b13041f970 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:45.707904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6c16-24ba-45f8-8947-01b13041f970 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:45.711462 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6c16-24ba-45f8-8947-01b13041f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:45.736689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-966d271e-764a-499a-8d6f-c782bd1186a0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:45.745814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-966d271e-764a-499a-8d6f-c782bd1186a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:45.753347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4e4d6c16-24ba-45f8-8947-01b13041f970 None None] MFA Rules not processed for user `494aa7af44284866b02f2a295b8b05aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:45.757375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e4d6c16-24ba-45f8-8947-01b13041f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:45.855879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1071/2119] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:45.879183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1049/2120] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:45.887905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10296aa-c093-4d87-b626-a5ce08a60ab0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:45.888275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10296aa-c093-4d87-b626-a5ce08a60ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:45.888648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10296aa-c093-4d87-b626-a5ce08a60ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:45.913774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f10296aa-c093-4d87-b626-a5ce08a60ab0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:45.916255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f10296aa-c093-4d87-b626-a5ce08a60ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:45.927109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1072/2121] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:37:45.931360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d271e-764a-499a-8d6f-c782bd1186a0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:45.945336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d271e-764a-499a-8d6f-c782bd1186a0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:45.953759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:45.953759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:45.953759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:45.953759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:45.954332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:37:45.955938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:45.957666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:45.988022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1050/2122] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:37:45.992373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10296aa-c093-4d87-b626-a5ce08a60ab0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:46.004860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10296aa-c093-4d87-b626-a5ce08a60ab0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:46.010294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:46.011368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:46.011368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:46.011677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:46.012320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:46.014503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:46.127075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1073/2123] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:37:45 2022] POST /identity/v3/projects => generated 400 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:37:46.130991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:46.141923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1feeed-80d5-4f2d-a633-f08465c3c525 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:46.163139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:46.216696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:46.218728 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:46.219289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:46.219968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:46.221782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:46.223769 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:46.318825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1051/2124] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:37:46 2022] POST /identity/v3/users => generated 301 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:37:46.321640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:46.339926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6a16a-8e92-4e43-90b1-a50209ffece7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:46.347165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:46.348096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:46.348564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:46.349011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:46.349883 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:46.351703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:46.363822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1074/2125] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:46 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:37:46.368070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:46.378264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e0adc-d278-426e-be00-cfe36fadfa99 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:46.383219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b133d-b7b2-4d34-aa7f-a02200ae1cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:46.383842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b133d-b7b2-4d34-aa7f-a02200ae1cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:46.384175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b133d-b7b2-4d34-aa7f-a02200ae1cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:46.384432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b133d-b7b2-4d34-aa7f-a02200ae1cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:46.385182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b133d-b7b2-4d34-aa7f-a02200ae1cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:46.386789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b133d-b7b2-4d34-aa7f-a02200ae1cd1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:46.399358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1052/2126] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:37:46 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:37:46.404648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:46.425590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc682404-591d-43b9-a20f-c2c8d5fbb415 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:46.430805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:46.434731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:46.434731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:46.434731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] PATH_INFO: `/v3/projects/c46311138a3844ddb8c8934e819da520/users/b6a6674919ce433eb55b0d3557993205/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:46.449278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46311138a3844ddb8c8934e819da520, user_id=b6a6674919ce433eb55b0d3557993205, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:46.452905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:46.554316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1075/2127] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:37:46 2022] PUT /identity/v3/projects/c46311138a3844ddb8c8934e819da520/users/b6a6674919ce433eb55b0d3557993205/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:46.565542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecdcc4-dfbc-4647-8671-3fe1093022be None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:46.565888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecdcc4-dfbc-4647-8671-3fe1093022be None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:46.566172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecdcc4-dfbc-4647-8671-3fe1093022be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:46.608731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-56ecdcc4-dfbc-4647-8671-3fe1093022be None None] MFA Rules not processed for user `b6a6674919ce433eb55b0d3557993205`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:46.613216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-56ecdcc4-dfbc-4647-8671-3fe1093022be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:46.723341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1053/2128] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:46.736594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:46.738543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:46.850931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ef62a-230c-43ef-af69-cf57736f1e2d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:46.855987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:46.856654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:46.856937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:46.857215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:46.869278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:46.871979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:46.898454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1076/2129] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:37:51.535143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd366-f98a-4505-8809-f7dcd7ef4cd2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:51.535626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd366-f98a-4505-8809-f7dcd7ef4cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:51.542822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd366-f98a-4505-8809-f7dcd7ef4cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:51.555325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5a3cd366-f98a-4505-8809-f7dcd7ef4cd2 None None] MFA Rules not processed for user `8c8e0d314c074eaa91f6b16c6777cf1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:51.556575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5a3cd366-f98a-4505-8809-f7dcd7ef4cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:51.616096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1054/2130] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:56.893815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=80 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:58.519393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:58.608029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4f2c-d7e1-42ec-8fe6-576b6faeec5f service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:58.612605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:58.613277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:58.613559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:58.618551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] PATH_INFO: `/v3/users/b6a6674919ce433eb55b0d3557993205` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:58.618551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a6674919ce433eb55b0d3557993205)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:58.618551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:58.844024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a6674919ce433eb55b0d3557993205'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:58.844721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a6674919ce433eb55b0d3557993205'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:58.851716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a6674919ce433eb55b0d3557993205'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:58.865051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a6674919ce433eb55b0d3557993205'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:58.994381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a6674919ce433eb55b0d3557993205'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:59.003812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1077/2131] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:37:58 2022] DELETE /identity/v3/users/b6a6674919ce433eb55b0d3557993205 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:59.008435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:59.008975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:59.009269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:59.029789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:37:59.031060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:59.066267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:37:59.103467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1055/2132] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:37:59.110659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:59.112491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:59.182751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461998b-bf10-429d-a4df-4b8aeb0c2aa7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:59.187050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4256c3b-72ba-4900-b12a-9ab80b0661e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:59.191703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4256c3b-72ba-4900-b12a-9ab80b0661e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:59.191703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4256c3b-72ba-4900-b12a-9ab80b0661e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:59.191703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4256c3b-72ba-4900-b12a-9ab80b0661e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:59.198297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4256c3b-72ba-4900-b12a-9ab80b0661e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:59.199735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4256c3b-72ba-4900-b12a-9ab80b0661e5 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:59.210427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1078/2133] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:37:59.749678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:37:59.751158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:37:59.763266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6984de-9bfd-453e-976f-bbc99023e9e0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:37:59.768964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:37:59.769520 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:59.769813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:59.770169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] PATH_INFO: `/v3/projects/c46311138a3844ddb8c8934e819da520` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:37:59.771308 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46311138a3844ddb8c8934e819da520)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:37:59.773875 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:37:59.895051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46311138a3844ddb8c8934e819da520', 'request_id': 'req-9b908239-20d9-4bce-836d-3d5a183121c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:59.970993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46311138a3844ddb8c8934e819da520', 'request_id': 'req-9b908239-20d9-4bce-836d-3d5a183121c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:59.977126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46311138a3844ddb8c8934e819da520', 'request_id': 'req-9b908239-20d9-4bce-836d-3d5a183121c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:59.988585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] The token cache is being invalidate because project c46311138a3844ddb8c8934e819da520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:37:59.993681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:59.993681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9b908239-20d9-4bce-836d-3d5a183121c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:37:59.993681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1056/2134] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:37:59 2022] DELETE /identity/v3/projects/c46311138a3844ddb8c8934e819da520 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:37:59.996250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758db8-c4a5-407d-89d5-811e36ed234b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:37:59.996781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758db8-c4a5-407d-89d5-811e36ed234b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:37:59.996901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758db8-c4a5-407d-89d5-811e36ed234b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.017931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5a758db8-c4a5-407d-89d5-811e36ed234b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:00.019180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5a758db8-c4a5-407d-89d5-811e36ed234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:00.037975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:00.082858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1079/2135] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:00.091942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab12ded-5759-4e13-8bb8-d4ac216d0040 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.092531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab12ded-5759-4e13-8bb8-d4ac216d0040 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.092865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab12ded-5759-4e13-8bb8-d4ac216d0040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.113730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5ab12ded-5759-4e13-8bb8-d4ac216d0040 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:00.115017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5ab12ded-5759-4e13-8bb8-d4ac216d0040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:00.124594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1057/2136] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:00.129012 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758db8-c4a5-407d-89d5-811e36ed234b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:00.141572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758db8-c4a5-407d-89d5-811e36ed234b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:00.150634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:00.151881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.152325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.152913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.154151 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:38:00.155216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:00.158155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:00.175196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1080/2137] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:38:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:38:00.179730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab12ded-5759-4e13-8bb8-d4ac216d0040 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:00.191272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab12ded-5759-4e13-8bb8-d4ac216d0040 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:00.200361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:00.201149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.201892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.202171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.202847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:00.207005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:00.304782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1058/2138] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:38:00 2022] POST /identity/v3/projects => generated 386 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:00.306181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:00.318588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3e5b-744b-4e9d-8df0-6bf33dedaf44 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:00.397386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:00.398022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.398304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.398588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.399244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:00.401201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:00.506243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1081/2139] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:38:00 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:00.509414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:00.520212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030364d-ec53-4fd6-8e36-bdd7b4af6f50 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:00.524955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:00.525595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.525908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.526192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.526908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:00.528564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:00.539528 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1059/2140] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:00 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:38:00.544148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:00.554779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5852da-0557-471c-84b2-707fb779918f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:00.561138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b0fe0-573d-4fa8-82b6-a1dc1004536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:00.561775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0fe0-573d-4fa8-82b6-a1dc1004536c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.562061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0fe0-573d-4fa8-82b6-a1dc1004536c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.562338 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0fe0-573d-4fa8-82b6-a1dc1004536c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.563062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b0fe0-573d-4fa8-82b6-a1dc1004536c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:00.564971 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b0fe0-573d-4fa8-82b6-a1dc1004536c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:00.577356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1082/2141] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:00 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:00.581552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:00.592794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab042-e091-4ee9-a326-6ea37a6f1d85 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:00.597515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:00.598181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.598468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.598744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] PATH_INFO: `/v3/projects/7c892ef56ae9488089386d1a733ae3c0/users/1f6f23b2e4134aa78862287f57323731/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.614400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c892ef56ae9488089386d1a733ae3c0, user_id=1f6f23b2e4134aa78862287f57323731, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:00.617205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:00.673883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1060/2142] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:00 2022] PUT /identity/v3/projects/7c892ef56ae9488089386d1a733ae3c0/users/1f6f23b2e4134aa78862287f57323731/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:00.679019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fb7b0-9a9c-4a7c-b984-1117252b79ec None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.679317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fb7b0-9a9c-4a7c-b984-1117252b79ec None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.679602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fb7b0-9a9c-4a7c-b984-1117252b79ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.700062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ae9fb7b0-9a9c-4a7c-b984-1117252b79ec None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:00.702323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ae9fb7b0-9a9c-4a7c-b984-1117252b79ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:00.767104 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1083/2143] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:00.776616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:00.778332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:00.837465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f475-7a97-456d-af39-4c9b624f624e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:00.841373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:00.842238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:00.845383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:00.845383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:00.850628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:00.853194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:00.854888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:00.865353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1061/2144] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:03.724536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:03.725546 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:03.726293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:03.783498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:03.785028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:03.836976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:03.881028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1084/2145] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:03.951092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:03.953466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:04.096326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5cc9-c644-4489-9dbb-f520799b4374 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:04.104201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:04.105443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:04.106104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:04.106533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:04.126757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:04.128739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:04.152670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1062/2146] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:38:09.131114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:09.133199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:09.240826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f6e08-2005-4fb2-b4da-7fca17c38ec0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:09.246648 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983aedd-4f6c-4ed0-a12d-2354bc951ee9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:09.248094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983aedd-4f6c-4ed0-a12d-2354bc951ee9 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:09.248563 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983aedd-4f6c-4ed0-a12d-2354bc951ee9 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:09.249039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983aedd-4f6c-4ed0-a12d-2354bc951ee9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:09.251081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3983aedd-4f6c-4ed0-a12d-2354bc951ee9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:09.267340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983aedd-4f6c-4ed0-a12d-2354bc951ee9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:09.269368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983aedd-4f6c-4ed0-a12d-2354bc951ee9 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:09.284729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1085/2147] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:10.072442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:10.077575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:10.089163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224992f-57a4-4f78-aa8d-57f858c11ec0 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:10.095063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:10.096800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:10.096800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:10.096800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] PATH_INFO: `/v3/users/9128bdfb7a09410ab22fd64cd27819b2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:10.098321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9128bdfb7a09410ab22fd64cd27819b2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:10.100137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:10.436964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9128bdfb7a09410ab22fd64cd27819b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:10.463024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9128bdfb7a09410ab22fd64cd27819b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:10.464262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9128bdfb7a09410ab22fd64cd27819b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:10.558890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9128bdfb7a09410ab22fd64cd27819b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:10.559764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9128bdfb7a09410ab22fd64cd27819b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:10.590607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1063/2148] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:38:10 2022] DELETE /identity/v3/users/9128bdfb7a09410ab22fd64cd27819b2 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:10.595734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:10.596358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:10.596358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:10.628175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:10.631069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:10.790956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1086/2149] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:10.826596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:10.829761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:11.102289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01baaf-f454-457d-b943-b78c1d1ae7eb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:11.112357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a895a5-763b-489a-ae5c-ea0e18c6502e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:11.113094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a895a5-763b-489a-ae5c-ea0e18c6502e service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:11.113386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a895a5-763b-489a-ae5c-ea0e18c6502e service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:11.113667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a895a5-763b-489a-ae5c-ea0e18c6502e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:11.194764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a895a5-763b-489a-ae5c-ea0e18c6502e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:11.203997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a895a5-763b-489a-ae5c-ea0e18c6502e service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:11.221756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:11.246614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1064/2150] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:12.863422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:12.866247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:12.886981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d189c-6ab7-47b8-881a-fc1f9c09734c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:12.909837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:12.909837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:12.911638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:12.912143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] PATH_INFO: `/v3/projects/115c7877ef274b759c5de1b5fd4b5267` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:12.916280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=115c7877ef274b759c5de1b5fd4b5267)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:12.925334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:13.175142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115c7877ef274b759c5de1b5fd4b5267', 'request_id': 'req-b72f97e7-1770-4352-b867-73d538571c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:13.189547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115c7877ef274b759c5de1b5fd4b5267', 'request_id': 'req-b72f97e7-1770-4352-b867-73d538571c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:13.205327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:13.212760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115c7877ef274b759c5de1b5fd4b5267', 'request_id': 'req-b72f97e7-1770-4352-b867-73d538571c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:13.360134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] The token cache is being invalidate because project 115c7877ef274b759c5de1b5fd4b5267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:38:13.361905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:13.364156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b72f97e7-1770-4352-b867-73d538571c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:13.368857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1087/2151] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:38:12 2022] DELETE /identity/v3/projects/115c7877ef274b759c5de1b5fd4b5267 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:13.374529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809085-ec6c-4058-bc04-87a522de9f51 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:13.375020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809085-ec6c-4058-bc04-87a522de9f51 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:13.375101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809085-ec6c-4058-bc04-87a522de9f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:13.439794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-68809085-ec6c-4058-bc04-87a522de9f51 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:13.447052 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-68809085-ec6c-4058-bc04-87a522de9f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:13.759950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1065/2152] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:13.768943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe10df-5175-42ba-8e95-937a47980430 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:13.769253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe10df-5175-42ba-8e95-937a47980430 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:13.769856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe10df-5175-42ba-8e95-937a47980430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:13.870989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c0fe10df-5175-42ba-8e95-937a47980430 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:13.872060 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0fe10df-5175-42ba-8e95-937a47980430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:13.932259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68809085-ec6c-4058-bc04-87a522de9f51 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:13.943746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68809085-ec6c-4058-bc04-87a522de9f51 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:13.945765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1088/2153] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:13.975409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e376099-c2a1-4b5b-ab1e-e42a7b01fc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:13.976065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e376099-c2a1-4b5b-ab1e-e42a7b01fc8e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:13.976354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e376099-c2a1-4b5b-ab1e-e42a7b01fc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:13.976666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e376099-c2a1-4b5b-ab1e-e42a7b01fc8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:13.977392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e376099-c2a1-4b5b-ab1e-e42a7b01fc8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:38:13.977707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e376099-c2a1-4b5b-ab1e-e42a7b01fc8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:13.979338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e376099-c2a1-4b5b-ab1e-e42a7b01fc8e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:14.039679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1066/2154] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:38:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:38:14.044668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe10df-5175-42ba-8e95-937a47980430 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:14.058502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe10df-5175-42ba-8e95-937a47980430 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:14.083852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:14.084621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:14.084924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:14.085213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:14.085916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:14.087896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:14.157592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:14.158076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:14.158399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:14.240915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] MFA Rules not processed for user `1f6f23b2e4134aa78862287f57323731`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:14.242405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:14.275923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1089/2155] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:38:14 2022] POST /identity/v3/projects => generated 372 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:14.279120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:14.437617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1067/2156] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:14.439237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:14.442571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:14.451683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f01d-fd4c-4022-a63c-58698a0d596c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:14.473715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:14.474640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:14.475087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:14.476860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:14.477432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281ae8b-944b-4f34-a285-674616a65160 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:14.478063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:14.481193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:14.495564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:14.496536 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:14.497590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:14.498132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] PATH_INFO: `/v3/users/2072a421cf924213902987e70da070c7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:14.499787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2072a421cf924213902987e70da070c7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:14.501996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:14.609102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1090/2157] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:38:14 2022] POST /identity/v3/users => generated 287 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:14.612730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:14.619135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:14.665432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a70735-d92c-41b5-ae9d-834009e66a03 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:14.678440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:14.679453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:14.680504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:14.680889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:14.682685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:14.686305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:14.802581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2072a421cf924213902987e70da070c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:14.814905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1091/2158] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:38:14 2022] POST /identity/v3/projects => generated 378 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:14.821657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:14.828862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2072a421cf924213902987e70da070c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:14.829754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2072a421cf924213902987e70da070c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:14.925559 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2072a421cf924213902987e70da070c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:14.926406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2072a421cf924213902987e70da070c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:14.932726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d2832-2ce7-498d-91b0-a7b932958d61 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:14.940226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:14.942320 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61801e6e-564f-4d68-8136-ea4165ca6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:14.943278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801e6e-564f-4d68-8136-ea4165ca6ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:14.943862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801e6e-564f-4d68-8136-ea4165ca6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:14.944339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801e6e-564f-4d68-8136-ea4165ca6ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:14.945683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61801e6e-564f-4d68-8136-ea4165ca6ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:14.957251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61801e6e-564f-4d68-8136-ea4165ca6ce8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:14.964286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1068/2159] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:38:14 2022] DELETE /identity/v3/users/2072a421cf924213902987e70da070c7 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:14.966586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:14.968236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:14.968478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1092/2160] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:14 2022] GET /identity/v3/roles => generated 1427 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:14.970055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c081f-ef69-4950-a058-4df25a91646f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:14.970729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c081f-ef69-4950-a058-4df25a91646f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:14.971158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c081f-ef69-4950-a058-4df25a91646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.000575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-195c081f-ef69-4950-a058-4df25a91646f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:15.002127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-195c081f-ef69-4950-a058-4df25a91646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:15.027653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:15.078204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5019f-d126-4666-8008-fbf9675a85c3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.087108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1093/2161] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:15.088652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c081f-ef69-4950-a058-4df25a91646f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.090329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-195c081f-ef69-4950-a058-4df25a91646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:15.101282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.102099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.102399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.103252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.103950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.107103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.111581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c081f-ef69-4950-a058-4df25a91646f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.118261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.121225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.121795 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.122533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.123594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.125689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.139614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1094/2162] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:15 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:15.141293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.144762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:15.252889 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1069/2163] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:38:14 2022] POST /identity/v3/users => generated 296 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:15.254115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.254685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335d321-d038-4438-8962-cfd72c42417b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.260350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.261208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.261709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.262098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.301509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.301509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.302147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b983b-7db2-4791-8aa5-3eab25963900 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.318569 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456509c-952f-4607-9f8b-4653bc249fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.319564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456509c-952f-4607-9f8b-4653bc249fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.320132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456509c-952f-4607-9f8b-4653bc249fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.320684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456509c-952f-4607-9f8b-4653bc249fb2 admin admin] PATH_INFO: `/v3/projects/23a35f3dedcb4c8d8c0a2f29b7b24a09/users/2fd948c627d2425fab2efd19cdf0d1d2/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.326894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1095/2164] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:15.330816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.360232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd4633-8c70-4705-9819-ec230323f455 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.364194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456509c-952f-4607-9f8b-4653bc249fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a35f3dedcb4c8d8c0a2f29b7b24a09, user_id=2fd948c627d2425fab2efd19cdf0d1d2, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.368669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456509c-952f-4607-9f8b-4653bc249fb2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.371695 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.372587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.373307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.373584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.378651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.382399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.411675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1096/2165] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:15 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:15.412961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.425713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57b394-750f-4c3c-8596-0dc0ba7a674c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.431448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.432285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.432851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.433252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] PATH_INFO: `/v3/projects/94a7db1ba4cf4b0d9a824c148c641109/users/5b2c1d60c19148a6ad2f893b0c00cdac/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.452338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1070/2166] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:15 2022] PUT /identity/v3/projects/23a35f3dedcb4c8d8c0a2f29b7b24a09/users/2fd948c627d2425fab2efd19cdf0d1d2/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:15.463725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.468003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.468003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.473113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a7db1ba4cf4b0d9a824c148c641109, user_id=5b2c1d60c19148a6ad2f893b0c00cdac, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.483017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.544656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] MFA Rules not processed for user `2fd948c627d2425fab2efd19cdf0d1d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:15.544656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:15.591987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1097/2167] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:15 2022] PUT /identity/v3/projects/94a7db1ba4cf4b0d9a824c148c641109/users/5b2c1d60c19148a6ad2f893b0c00cdac/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:15.593384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.755571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1071/2168] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:15.767511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.769289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:15.800436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b115aa8-aa48-48e0-ac28-6236e4161cb4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.811910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.813100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.813725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.814249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.816141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.818474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.835387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1098/2169] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:15 2022] GET /identity/v3/roles => generated 1427 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:15.837029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:15.850236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ca3bf-40d9-45df-aea1-43f4c0ba41d9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:15.857621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:15.858675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:15.859193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:15.859876 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] PATH_INFO: `/v3/projects/94a7db1ba4cf4b0d9a824c148c641109/users/5b2c1d60c19148a6ad2f893b0c00cdac/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:15.862473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a7db1ba4cf4b0d9a824c148c641109, user_id=5b2c1d60c19148a6ad2f893b0c00cdac, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:15.865551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:15.898531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1099/2170] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:15 2022] PUT /identity/v3/projects/94a7db1ba4cf4b0d9a824c148c641109/users/5b2c1d60c19148a6ad2f893b0c00cdac/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:15.913608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:15.937320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0395ef1-ff02-47e8-81b2-7f1cc73439fb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:16.060387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b54d5-3187-426f-b193-86270f462b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:16.061081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b54d5-3187-426f-b193-86270f462b68 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:16.061370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b54d5-3187-426f-b193-86270f462b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:16.061648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b54d5-3187-426f-b193-86270f462b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:16.210017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b54d5-3187-426f-b193-86270f462b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:16.217455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b54d5-3187-426f-b193-86270f462b68 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:16.253457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1072/2171] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:16.320112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:16.322674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:16.328711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:16.518323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e55646-0887-4730-8ee6-2b8ba800f12e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:16.529911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:16.533984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:16.533984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:16.533984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] PATH_INFO: `/v3/projects/e045fe1c61644fde9dabd25af5f36970` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:16.535006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e045fe1c61644fde9dabd25af5f36970)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:16.537173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:16.808433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e045fe1c61644fde9dabd25af5f36970', 'request_id': 'req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:16.822911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e045fe1c61644fde9dabd25af5f36970', 'request_id': 'req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:16.919356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e045fe1c61644fde9dabd25af5f36970', 'request_id': 'req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:17.636712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] The token cache is being invalidate because project e045fe1c61644fde9dabd25af5f36970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:38:17.636712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:17.636712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-88d39ac9-acd7-4dc9-a62e-82025f4c2e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:17.638528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1100/2172] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:38:16 2022] DELETE /identity/v3/projects/e045fe1c61644fde9dabd25af5f36970 => generated 0 bytes in 1319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:17.644091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a393-c015-41d2-856d-d7cada61b951 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:17.644695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a393-c015-41d2-856d-d7cada61b951 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:17.644822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a393-c015-41d2-856d-d7cada61b951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:17.700101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-30a6a393-c015-41d2-856d-d7cada61b951 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:17.702359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-30a6a393-c015-41d2-856d-d7cada61b951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:17.874686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1073/2173] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:17.886698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d67d2c-57a4-43ac-bb42-a38a16c7e924 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:17.887076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d67d2c-57a4-43ac-bb42-a38a16c7e924 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:17.887455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d67d2c-57a4-43ac-bb42-a38a16c7e924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:17.917727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a9d67d2c-57a4-43ac-bb42-a38a16c7e924 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:17.923336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a9d67d2c-57a4-43ac-bb42-a38a16c7e924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:17.951471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6a393-c015-41d2-856d-d7cada61b951 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:17.959277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1101/2174] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:17.963935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6a393-c015-41d2-856d-d7cada61b951 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:17.976949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:17.977998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:17.978481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:17.978941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:17.980123 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:38:17.980616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:17.983257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:18.005504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d67d2c-57a4-43ac-bb42-a38a16c7e924 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:18.018611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1074/2175] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:38:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:38:18.027402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d67d2c-57a4-43ac-bb42-a38a16c7e924 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:18.034155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:18.041910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:18.041910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:18.041910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:18.041910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:18.041910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:18.254244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1102/2176] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:38:18 2022] POST /identity/v3/projects => generated 378 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:18.258323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:18.289888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021afe8-61ff-4d6f-b422-c50d3d7f608c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:18.427755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:18.487092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:18.491156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:18.491687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:18.492135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:18.494586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:18.500032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:18.694777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1075/2177] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:38:18 2022] POST /identity/v3/users => generated 290 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:18.696902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:18.720960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873727-59c3-4dd6-9999-003c2523cccb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:18.743567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:18.747916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:18.747916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:18.747916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:18.747916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:18.753297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:18.812990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1103/2178] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:18 2022] GET /identity/v3/roles => generated 1427 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:18.819937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:18.835288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f66199-da61-4a17-aed9-ae0db5d2a0fb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:18.851086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb6ee2-f799-49f6-9984-4adc5122746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:18.851996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb6ee2-f799-49f6-9984-4adc5122746e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:18.852411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb6ee2-f799-49f6-9984-4adc5122746e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:18.853013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb6ee2-f799-49f6-9984-4adc5122746e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:18.857941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb6ee2-f799-49f6-9984-4adc5122746e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:18.857941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb6ee2-f799-49f6-9984-4adc5122746e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:18.891985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1076/2179] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:18 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:18.893108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:18.908109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2063bf-b296-4643-b4de-bca871261124 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:18.920244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:18.921286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:18.921739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:18.922179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] PATH_INFO: `/v3/projects/3e5dd905492343709b710fabd8a3d24f/users/39f0a10b47b84bd1bd8ff8177362d791/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:18.949378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5dd905492343709b710fabd8a3d24f, user_id=39f0a10b47b84bd1bd8ff8177362d791, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:18.955712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:19.069602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1104/2180] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:18 2022] PUT /identity/v3/projects/3e5dd905492343709b710fabd8a3d24f/users/39f0a10b47b84bd1bd8ff8177362d791/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:19.077168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7106e36-6bfd-4bfb-95cb-5a288ea05ee9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:19.078376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7106e36-6bfd-4bfb-95cb-5a288ea05ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:19.078922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7106e36-6bfd-4bfb-95cb-5a288ea05ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:19.200725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e7106e36-6bfd-4bfb-95cb-5a288ea05ee9 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:19.203925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e7106e36-6bfd-4bfb-95cb-5a288ea05ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:19.405161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1077/2181] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:19.412754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:19.417924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:19.498751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:19.569927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e6825-d5b1-4cce-acb1-d53e835c8119 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:19.582110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:19.582859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:19.583154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:19.583438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:19.606027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:19.607676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:19.634585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1105/2182] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:29.388849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:29.764080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=13 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:29.930582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98287099-186c-4f8d-b517-584e92dbe8ad None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:29.931006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98287099-186c-4f8d-b517-584e92dbe8ad None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:29.934335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98287099-186c-4f8d-b517-584e92dbe8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:29.974524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-98287099-186c-4f8d-b517-584e92dbe8ad None None] MFA Rules not processed for user `5b2c1d60c19148a6ad2f893b0c00cdac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:29.976132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-98287099-186c-4f8d-b517-584e92dbe8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:30.098815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1078/2183] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:30.112733 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:30.114378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:30.295313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1722a-798a-4860-8169-ae9fb28010b1 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:30.305878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:30.310644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:30.311081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:30.311477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:30.315651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:30.553845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:30.559663 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:30.608915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1106/2184] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:31.015613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:31.015613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:31.015613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:31.058086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:31.060747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:31.199783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1079/2185] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:31.216290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:31.218699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:31.450722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a70f6-d0ff-4633-95ac-751ed7b03af1 service glance] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:31.468395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f169c49-a5c7-408e-8a5f-f84cb72951fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:31.469194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169c49-a5c7-408e-8a5f-f84cb72951fd service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:31.469926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169c49-a5c7-408e-8a5f-f84cb72951fd service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:31.470394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169c49-a5c7-408e-8a5f-f84cb72951fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:31.484187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f169c49-a5c7-408e-8a5f-f84cb72951fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:31.486739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f169c49-a5c7-408e-8a5f-f84cb72951fd service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:31.495362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1107/2186] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:38:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:38:33.137613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:33.145306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:33.181350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec3c0f-8f65-44f8-8795-c6a0a144984f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:33.190112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:33.191560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:33.192027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:33.192613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] PATH_INFO: `/v3/users/2fd948c627d2425fab2efd19cdf0d1d2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:33.194572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd948c627d2425fab2efd19cdf0d1d2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:33.197222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:33.533929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd948c627d2425fab2efd19cdf0d1d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:33.549729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:33.550270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:33.550503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:33.555391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd948c627d2425fab2efd19cdf0d1d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:33.556609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd948c627d2425fab2efd19cdf0d1d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:33.606345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] MFA Rules not processed for user `39f0a10b47b84bd1bd8ff8177362d791`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:33.608629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:33.672949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:33.677658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd948c627d2425fab2efd19cdf0d1d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:33.678533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e7ba28-e3a1-4bb7-9cd9-d697abfa2532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd948c627d2425fab2efd19cdf0d1d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:33.724492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1080/2187] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:38:33 2022] DELETE /identity/v3/users/2fd948c627d2425fab2efd19cdf0d1d2 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:33.729357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:33.729651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:33.729951 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:33.749330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:33.775658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:33.782283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1108/2188] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:33.782733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:33.783285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:33.787328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:33.827127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db18b-84b0-4ff8-8b84-77314548eddc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:33.947676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:33.951666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:33.951666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:33.951666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:33.952277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:33.961905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:33.991817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1081/2189] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:34.006999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:34.008542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:34.130111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1109/2190] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:38:33 2022] POST /identity/v3/projects => generated 378 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:38:34.135166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:34.156654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ec33d-a8f6-4261-9eb9-f8ff36791ad3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:34.289871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:34.290810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:34.291627 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:34.292077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:34.312873 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93871aea-a3d6-4e39-ac4b-c081a7aa2252 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:34.314078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:34.319740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:34.319968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:34.320506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:34.320806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:34.321137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:34.321781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:34.325448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:34.353908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1082/2191] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:34.438102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1110/2192] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:38:34 2022] POST /identity/v3/users => generated 296 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:34.442736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:34.460436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6166d8-75b6-4e25-a79d-dfd6be485e0a service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:34.467401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:34.468287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:34.468747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:34.469101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:34.470151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:34.472188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:34.495233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1083/2193] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:34 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:38:34.500274 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:34.519294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c8058-32c3-476d-ab93-0688f1ec22f8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:34.536315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:34.537059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:34.537348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:34.537632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] PATH_INFO: `/v3/projects/b445144679f543c9af86651752925487/users/7c87d7f195c543d58ea5975f059f262b/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:34.612951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b445144679f543c9af86651752925487, user_id=7c87d7f195c543d58ea5975f059f262b, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:34.617621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:34.704662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1111/2194] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:34 2022] PUT /identity/v3/projects/b445144679f543c9af86651752925487/users/7c87d7f195c543d58ea5975f059f262b/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:34.706496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:34.742870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1919d0-b359-4d49-91af-121893d5762f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:34.876661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:34.901292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:34.902014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:34.902577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:34.902996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:34.903828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:34.906451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:34.926941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1084/2195] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:34 2022] GET /identity/v3/roles => generated 1427 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:34.928492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:34.950493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beaf1a0-ae16-47b4-b41c-8ce4d8e3fba7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:34.966167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9ebb7-66ff-433b-8d64-2b1eb1579a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:34.973473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ebb7-66ff-433b-8d64-2b1eb1579a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:34.973473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ebb7-66ff-433b-8d64-2b1eb1579a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:34.973473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ebb7-66ff-433b-8d64-2b1eb1579a33 admin admin] PATH_INFO: `/v3/projects/b445144679f543c9af86651752925487/users/7c87d7f195c543d58ea5975f059f262b/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:34.973473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9ebb7-66ff-433b-8d64-2b1eb1579a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b445144679f543c9af86651752925487, user_id=7c87d7f195c543d58ea5975f059f262b, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:34.979344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9ebb7-66ff-433b-8d64-2b1eb1579a33 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:35.001810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1112/2196] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:34 2022] PUT /identity/v3/projects/b445144679f543c9af86651752925487/users/7c87d7f195c543d58ea5975f059f262b/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:38:35.104661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:35.104661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:35.173352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975388f6-7758-4cb2-8bbe-78535543dcf2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:35.290987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:35.292020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:35.294729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:35.295025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] PATH_INFO: `/v3/projects/23a35f3dedcb4c8d8c0a2f29b7b24a09` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:35.296348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a35f3dedcb4c8d8c0a2f29b7b24a09)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:35.300953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:35.546821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a35f3dedcb4c8d8c0a2f29b7b24a09', 'request_id': 'req-7a15a26c-21c0-4b16-a2f6-57e88db994b9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:35.620379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a35f3dedcb4c8d8c0a2f29b7b24a09', 'request_id': 'req-7a15a26c-21c0-4b16-a2f6-57e88db994b9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:35.628752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a35f3dedcb4c8d8c0a2f29b7b24a09', 'request_id': 'req-7a15a26c-21c0-4b16-a2f6-57e88db994b9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:35.659260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] The token cache is being invalidate because project 23a35f3dedcb4c8d8c0a2f29b7b24a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:38:35.659660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:35.660358 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a15a26c-21c0-4b16-a2f6-57e88db994b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:38:35.663365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1085/2197] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:38:35 2022] DELETE /identity/v3/projects/23a35f3dedcb4c8d8c0a2f29b7b24a09 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:38:35.682258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4240-530b-489c-b195-a05ef847ea93 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:35.682258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4240-530b-489c-b195-a05ef847ea93 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:35.682258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4240-530b-489c-b195-a05ef847ea93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:35.735570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-94bb4240-530b-489c-b195-a05ef847ea93 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:35.737111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-94bb4240-530b-489c-b195-a05ef847ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:35.753933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:35.926741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1113/2198] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:35.934189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dde6d-38d9-4c7d-be08-71cddbc50956 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:35.934189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dde6d-38d9-4c7d-be08-71cddbc50956 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:35.934189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dde6d-38d9-4c7d-be08-71cddbc50956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:36.035954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-446dde6d-38d9-4c7d-be08-71cddbc50956 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:36.047653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-446dde6d-38d9-4c7d-be08-71cddbc50956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:36.075317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1086/2199] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:36.081946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4240-530b-489c-b195-a05ef847ea93 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:36.092101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4240-530b-489c-b195-a05ef847ea93 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:36.100986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:36.101640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:36.101927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:36.102208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:36.103094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:38:36.103415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:36.106628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:36.129169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1114/2200] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:38:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:38:36.133674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dde6d-38d9-4c7d-be08-71cddbc50956 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:36.146347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dde6d-38d9-4c7d-be08-71cddbc50956 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:36.152583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:36.152583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:36.152583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:36.153104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:36.154589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:36.156146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:36.262964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1087/2201] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:38:36 2022] POST /identity/v3/projects => generated 376 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:36.270208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:36.283111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e0bf5-09ea-48c9-96ca-bde9ee529d8a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:36.452040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:36.452719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:36.453007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:36.453285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:36.453974 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:36.456116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:36.555187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1115/2202] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:38:36 2022] POST /identity/v3/users => generated 289 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:38:36.559945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:36.578471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673a0b-407f-4ffa-bbf9-37fb9e671d52 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:36.596630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:36.596630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:36.596630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:36.596630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:36.597073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:36.598378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:36.611759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1088/2203] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:36 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:36.615692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:36.633270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a02490-901a-44bb-b277-119ef4534424 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:36.649300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020c4e5-bc9b-4736-b9f9-54eb190833a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:36.652725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c4e5-bc9b-4736-b9f9-54eb190833a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:36.652725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c4e5-bc9b-4736-b9f9-54eb190833a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:36.652725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c4e5-bc9b-4736-b9f9-54eb190833a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:36.654689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020c4e5-bc9b-4736-b9f9-54eb190833a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:36.656770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020c4e5-bc9b-4736-b9f9-54eb190833a7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:36.679303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1116/2204] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:38:36 2022] GET /identity/v3/roles => generated 1427 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:38:36.679808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:36.710129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd5f16-1891-4d5c-9f58-88c59dac3597 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:36.720186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:36.722380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:36.722689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:36.722982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] PATH_INFO: `/v3/projects/d432a53d4fc44e8e9e557f7e3bca2c47/users/c746449a143140fc87ee685c4fc5f3ea/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:36.749707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d432a53d4fc44e8e9e557f7e3bca2c47, user_id=c746449a143140fc87ee685c4fc5f3ea, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:36.757010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:36.919420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1089/2205] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:38:36 2022] PUT /identity/v3/projects/d432a53d4fc44e8e9e557f7e3bca2c47/users/c746449a143140fc87ee685c4fc5f3ea/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:38:36.934833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48fe57-87f7-4eb6-8776-58bd1646a264 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:36.935821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48fe57-87f7-4eb6-8776-58bd1646a264 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:36.936333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48fe57-87f7-4eb6-8776-58bd1646a264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:37.027006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0e48fe57-87f7-4eb6-8776-58bd1646a264 None None] MFA Rules not processed for user `c746449a143140fc87ee685c4fc5f3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:37.031111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0e48fe57-87f7-4eb6-8776-58bd1646a264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:37.189487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1117/2206] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:37.196244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:37.198057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:37.277397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:37.418299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed6f03-20a5-4a06-950b-e624877e4f1f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:37.439095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd358d9-da7d-4b4e-9d0d-ecb7ff32dc7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:37.441054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd358d9-da7d-4b4e-9d0d-ecb7ff32dc7f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:37.441596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd358d9-da7d-4b4e-9d0d-ecb7ff32dc7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:37.442094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd358d9-da7d-4b4e-9d0d-ecb7ff32dc7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:37.487264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd358d9-da7d-4b4e-9d0d-ecb7ff32dc7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:37.489454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd358d9-da7d-4b4e-9d0d-ecb7ff32dc7f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:37.505101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1090/2207] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:38.141032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3801f-13be-4ae0-9c23-bd56734b7a75 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:38.141544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3801f-13be-4ae0-9c23-bd56734b7a75 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:38.141838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3801f-13be-4ae0-9c23-bd56734b7a75 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:38.143304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1118/2208] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:38:38 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:38:38.160136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed0985-9178-4dca-b9f6-3e7218a8d993 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:38.160714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed0985-9178-4dca-b9f6-3e7218a8d993 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:38.161156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed0985-9178-4dca-b9f6-3e7218a8d993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:38.221457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-76ed0985-9178-4dca-b9f6-3e7218a8d993 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:38.223703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-76ed0985-9178-4dca-b9f6-3e7218a8d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:38.432952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1091/2209] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:38.445068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3801f-13be-4ae0-9c23-bd56734b7a75 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:38.447070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f4d3801f-13be-4ae0-9c23-bd56734b7a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:38.492067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:38.618450 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3801f-13be-4ae0-9c23-bd56734b7a75 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:38.626911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:38.627566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:38.628089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:38.628766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:38.646251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:38.649293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:38.692698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1119/2210] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:38.983088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462df18c-0424-4d77-a656-3e857489a7c7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:38.983277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462df18c-0424-4d77-a656-3e857489a7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:38.984185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462df18c-0424-4d77-a656-3e857489a7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:39.018640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-462df18c-0424-4d77-a656-3e857489a7c7 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:39.020004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-462df18c-0424-4d77-a656-3e857489a7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:39.146088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1092/2211] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:39.167653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:39.169759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:39.328086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304fc0e-39dc-4304-b05f-9be522375d22 service glance] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:39.340172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f292cb9-c8cf-4fcc-af2c-2b3eca480a13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:39.340824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f292cb9-c8cf-4fcc-af2c-2b3eca480a13 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:39.341111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f292cb9-c8cf-4fcc-af2c-2b3eca480a13 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:39.341432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f292cb9-c8cf-4fcc-af2c-2b3eca480a13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:39.357065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f292cb9-c8cf-4fcc-af2c-2b3eca480a13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:39.358782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f292cb9-c8cf-4fcc-af2c-2b3eca480a13 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:39.375827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1120/2212] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:38:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:38:39.698124 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59745-e637-4387-9c5d-096766f0142e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:39.699930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59745-e637-4387-9c5d-096766f0142e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:39.699930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59745-e637-4387-9c5d-096766f0142e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:39.734399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-15d59745-e637-4387-9c5d-096766f0142e None None] MFA Rules not processed for user `c746449a143140fc87ee685c4fc5f3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:39.739935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15d59745-e637-4387-9c5d-096766f0142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:39.760656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1093/2213] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:45.185946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671885f1-67c6-480e-a115-54961446e06f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:45.186504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671885f1-67c6-480e-a115-54961446e06f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:45.186504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671885f1-67c6-480e-a115-54961446e06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:45.293471 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-671885f1-67c6-480e-a115-54961446e06f None None] MFA Rules not processed for user `7c87d7f195c543d58ea5975f059f262b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:45.295085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-671885f1-67c6-480e-a115-54961446e06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:45.474951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1121/2214] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:38:45.615968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d59745-e637-4387-9c5d-096766f0142e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:45.621289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15d59745-e637-4387-9c5d-096766f0142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:45.647993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d59745-e637-4387-9c5d-096766f0142e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:45.655111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec857d6e-5103-4a0b-a2cc-f987c713464e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:45.655845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec857d6e-5103-4a0b-a2cc-f987c713464e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:45.656136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec857d6e-5103-4a0b-a2cc-f987c713464e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:45.656411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec857d6e-5103-4a0b-a2cc-f987c713464e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:45.658694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ec857d6e-5103-4a0b-a2cc-f987c713464e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:45.736172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:45.747379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec857d6e-5103-4a0b-a2cc-f987c713464e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:45.754179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec857d6e-5103-4a0b-a2cc-f987c713464e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:45.761424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1094/2215] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:51.259880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e2ce-d926-4b6a-a5e2-278a2f65aa98 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:51.262186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e2ce-d926-4b6a-a5e2-278a2f65aa98 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:51.262796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e2ce-d926-4b6a-a5e2-278a2f65aa98 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:51.265153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1122/2216] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:38:51 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:38:51.270126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44967d-4d2e-47ca-8736-ac3f38256b3b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:51.271134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44967d-4d2e-47ca-8736-ac3f38256b3b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:51.271760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44967d-4d2e-47ca-8736-ac3f38256b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:51.339822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4d44967d-4d2e-47ca-8736-ac3f38256b3b None None] MFA Rules not processed for user `c6ecc741022f4d6189f4d3f19b4cdbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:51.341091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4d44967d-4d2e-47ca-8736-ac3f38256b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:51.528894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1095/2217] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:51.569965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8e2ce-d926-4b6a-a5e2-278a2f65aa98 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:51.574306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90c8e2ce-d926-4b6a-a5e2-278a2f65aa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:51.603295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0980f-1b66-4694-aee6-7a58396ba1de None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:51.604404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0980f-1b66-4694-aee6-7a58396ba1de None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:51.604784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0980f-1b66-4694-aee6-7a58396ba1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:51.609431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8e2ce-d926-4b6a-a5e2-278a2f65aa98 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:51.618881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d310a-eb6d-4d3a-ae1b-29863811c447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:51.620625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d310a-eb6d-4d3a-ae1b-29863811c447 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:51.620921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d310a-eb6d-4d3a-ae1b-29863811c447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:51.632434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d310a-eb6d-4d3a-ae1b-29863811c447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:51.647687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c1f0980f-1b66-4694-aee6-7a58396ba1de None None] MFA Rules not processed for user `1f6f23b2e4134aa78862287f57323731`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:51.649497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d310a-eb6d-4d3a-ae1b-29863811c447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:51.651629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d310a-eb6d-4d3a-ae1b-29863811c447 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:51.653114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c1f0980f-1b66-4694-aee6-7a58396ba1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:51.677085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1123/2218] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:38:51.811198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1096/2219] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:51.879340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4db3a-8e07-4beb-9e9b-89846e81051c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:51.879798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4db3a-8e07-4beb-9e9b-89846e81051c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:51.880208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4db3a-8e07-4beb-9e9b-89846e81051c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:51.886236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1124/2220] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:38:51 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:38:51.911250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d2e88-8033-438d-aa13-10d2bf196e1a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:51.911938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d2e88-8033-438d-aa13-10d2bf196e1a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:51.912446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d2e88-8033-438d-aa13-10d2bf196e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:51.968549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-584d2e88-8033-438d-aa13-10d2bf196e1a None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:38:51.972644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-584d2e88-8033-438d-aa13-10d2bf196e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:38:52.013580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1097/2221] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:38:52.022730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4db3a-8e07-4beb-9e9b-89846e81051c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:38:52.052181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4db3a-8e07-4beb-9e9b-89846e81051c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:38:52.063397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:38:52.064417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:38:52.065166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:38:52.065611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:38:52.092693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:38:52.094483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:38:52.107790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:38:52.122759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1125/2222] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:39:01.950035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=58 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:02.157312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:07.495295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea988f33-5f63-4346-b862-f03e2439d47c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:07.499659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea988f33-5f63-4346-b862-f03e2439d47c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:07.499659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea988f33-5f63-4346-b862-f03e2439d47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:07.515866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ea988f33-5f63-4346-b862-f03e2439d47c None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:07.517158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ea988f33-5f63-4346-b862-f03e2439d47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:07.527848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1098/2223] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:39:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:39:07.541436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:07.554196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65645bde-2bce-42ce-83cd-50c34d041057 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:07.559750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:07.560385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:07.560738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:07.561025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:07.570047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:07.572348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:07.581345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1126/2224] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:39:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:39:08.695575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:08.696092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:08.696914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:08.735993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:08.737259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:08.749704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1099/2225] 10.222.0.128 () {58 vars in 972 bytes} [Tue Jul 26 15:39:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:39:08.766543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:08.783083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2e4ca-26bd-402e-8bd1-038b37631b67 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:08.788750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:08.789654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:08.789951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:08.790803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:08.800187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:08.801863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:08.812364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1127/2226] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:39:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:39:13.914841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:13.920645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:14.033142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3046bf0-20bd-464a-8ac0-6f65961e63b5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:14.039729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:14.040878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:14.041315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:14.041766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] PATH_INFO: `/v3/users/8c8e0d314c074eaa91f6b16c6777cf1e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:14.043189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c8e0d314c074eaa91f6b16c6777cf1e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:14.045436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:14.205523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c8e0d314c074eaa91f6b16c6777cf1e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:14.213304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c8e0d314c074eaa91f6b16c6777cf1e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:14.214227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c8e0d314c074eaa91f6b16c6777cf1e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:14.257576 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c8e0d314c074eaa91f6b16c6777cf1e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:14.260920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e55dfd2d-c0bd-4274-bc25-9e66282f8b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c8e0d314c074eaa91f6b16c6777cf1e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:14.299193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1100/2227] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:39:13 2022] DELETE /identity/v3/users/8c8e0d314c074eaa91f6b16c6777cf1e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:39:14.639316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:14.659510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f89e2b-7376-4a75-8da9-84a2da4f1e51 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:14.749987 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:14.751101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:14.751586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:14.752148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] PATH_INFO: `/v3/projects/eb5cc7e509e84773bec86203332e38af` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:14.753849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5cc7e509e84773bec86203332e38af)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:14.756688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:14.882755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5cc7e509e84773bec86203332e38af', 'request_id': 'req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:14.890765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5cc7e509e84773bec86203332e38af', 'request_id': 'req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:14.898222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5cc7e509e84773bec86203332e38af', 'request_id': 'req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:15.886538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] The token cache is being invalidate because project eb5cc7e509e84773bec86203332e38af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:39:15.887408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:15.887702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-439dc3e4-7ed8-4586-bc1a-fefb3f51ef67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:15.891455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1128/2228] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:39:14 2022] DELETE /identity/v3/projects/eb5cc7e509e84773bec86203332e38af => generated 0 bytes in 1251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:39:15.919135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ddc4-dd16-4541-a4ab-a376afd92f2d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:15.920644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ddc4-dd16-4541-a4ab-a376afd92f2d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:15.920644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ddc4-dd16-4541-a4ab-a376afd92f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:15.942271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ff53ddc4-dd16-4541-a4ab-a376afd92f2d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:15.943596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ff53ddc4-dd16-4541-a4ab-a376afd92f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:16.026204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1101/2229] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:39:16.028643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e13e8-010f-42e2-ad64-e3d21524f0fb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.028943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e13e8-010f-42e2-ad64-e3d21524f0fb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.029253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e13e8-010f-42e2-ad64-e3d21524f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.058228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f87e13e8-010f-42e2-ad64-e3d21524f0fb None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:16.060739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f87e13e8-010f-42e2-ad64-e3d21524f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:16.078360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1129/2230] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:39:16.082907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53ddc4-dd16-4541-a4ab-a376afd92f2d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:16.103325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53ddc4-dd16-4541-a4ab-a376afd92f2d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:16.110602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:16.111248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.111533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.111819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.113090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:39:16.113590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:16.122237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:16.125551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:16.145651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1102/2231] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:39:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:39:16.146916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e13e8-010f-42e2-ad64-e3d21524f0fb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:16.161412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e13e8-010f-42e2-ad64-e3d21524f0fb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:16.170042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:16.176860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.177292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.177586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.178605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:16.186845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:16.313825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1130/2232] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:39:16 2022] POST /identity/v3/projects => generated 372 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:39:16.317499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:16.334122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd415a-3bec-46f0-893d-21a76b2587df admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:16.470842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:16.471780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.472201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.472792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.473577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:16.476133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:16.590529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1103/2233] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:39:16 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:39:16.594669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:16.607164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6021f-3581-403e-9bfd-159b1ebc67a6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:16.611966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:16.612706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.613055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.613561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.614341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:16.616018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:16.629609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1131/2234] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:39:16 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:39:16.635371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:16.646543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87987-c3f0-4122-b93b-dfb8f893b632 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:16.658466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bf396-d384-4fd5-89d0-b8f68effc8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:16.659446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bf396-d384-4fd5-89d0-b8f68effc8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.659873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bf396-d384-4fd5-89d0-b8f68effc8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.660824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bf396-d384-4fd5-89d0-b8f68effc8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.661922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bf396-d384-4fd5-89d0-b8f68effc8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:16.664067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bf396-d384-4fd5-89d0-b8f68effc8e3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:16.688854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1104/2235] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:39:16 2022] GET /identity/v3/roles => generated 1427 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:39:16.693463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:16.706858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d43eaa-db3c-4f95-b109-e449b41ae695 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:16.711918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:16.712797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.713160 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.713515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] PATH_INFO: `/v3/projects/7a82efb520f34fe684bffa4a8692180d/users/0c1150e967a94f9a98c90a226358aff7/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.739719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a82efb520f34fe684bffa4a8692180d, user_id=0c1150e967a94f9a98c90a226358aff7, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:16.743989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:16.814849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:16.827154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1132/2236] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:39:16 2022] PUT /identity/v3/projects/7a82efb520f34fe684bffa4a8692180d/users/0c1150e967a94f9a98c90a226358aff7/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:39:16.831824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6d49d-67bb-4f76-8f74-705d6a30a352 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:16.832182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6d49d-67bb-4f76-8f74-705d6a30a352 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:16.832505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6d49d-67bb-4f76-8f74-705d6a30a352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:16.869390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-86f6d49d-67bb-4f76-8f74-705d6a30a352 None None] MFA Rules not processed for user `0c1150e967a94f9a98c90a226358aff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:16.870758 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-86f6d49d-67bb-4f76-8f74-705d6a30a352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:16.948384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1105/2237] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:39:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:39:16.960536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:16.964146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:17.042713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c133a-ab38-4dcd-86ba-e5c0979967c0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:17.047203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66227ed7-5477-4064-afb5-792f592f850d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:17.048039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66227ed7-5477-4064-afb5-792f592f850d service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:17.048540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66227ed7-5477-4064-afb5-792f592f850d service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:17.049084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66227ed7-5477-4064-afb5-792f592f850d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:17.060442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66227ed7-5477-4064-afb5-792f592f850d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:17.062294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66227ed7-5477-4064-afb5-792f592f850d service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:17.079781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1133/2238] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:39:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:39:17.659704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:17.660392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:17.660392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:17.681614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] MFA Rules not processed for user `39f0a10b47b84bd1bd8ff8177362d791`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:17.683419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:17.748299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1106/2239] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:39:18.806315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd8fe2-a251-4cc8-b3c7-737405087c6b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:18.806939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd8fe2-a251-4cc8-b3c7-737405087c6b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:18.806939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd8fe2-a251-4cc8-b3c7-737405087c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:18.833074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0cfd8fe2-a251-4cc8-b3c7-737405087c6b None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:18.834434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0cfd8fe2-a251-4cc8-b3c7-737405087c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:18.926701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1134/2240] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:39:18.944867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:18.948230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:18.972839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:19.067979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7ee3c-ba24-49cb-b8b7-9b95c16ccaa6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:19.074334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec615bcd-1c30-4c06-bd53-cf40f97f40fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:19.075111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615bcd-1c30-4c06-bd53-cf40f97f40fa service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:19.075370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615bcd-1c30-4c06-bd53-cf40f97f40fa service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:19.075778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615bcd-1c30-4c06-bd53-cf40f97f40fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:19.114253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec615bcd-1c30-4c06-bd53-cf40f97f40fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:19.116082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec615bcd-1c30-4c06-bd53-cf40f97f40fa service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:19.137370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1107/2241] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:39:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:39:28.920888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=51 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:29.131032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=29 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:47.666829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd8fe2-a251-4cc8-b3c7-737405087c6b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:47.731157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd8fe2-a251-4cc8-b3c7-737405087c6b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:47.736760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:47.737568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:47.737941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:47.738308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] PATH_INFO: `/v3/users/0c1150e967a94f9a98c90a226358aff7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:47.739654 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1150e967a94f9a98c90a226358aff7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:47.742077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:47.965700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1150e967a94f9a98c90a226358aff7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:47.966779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1150e967a94f9a98c90a226358aff7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:47.976348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1150e967a94f9a98c90a226358aff7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:47.993193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1150e967a94f9a98c90a226358aff7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:48.055033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1150e967a94f9a98c90a226358aff7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:48.065504 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1135/2242] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:39:47 2022] DELETE /identity/v3/users/0c1150e967a94f9a98c90a226358aff7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:39:48.068044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:48.068634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:48.068634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:48.089306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:39:48.090544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:48.183919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1108/2243] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:39:48.194602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:48.196095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:48.259886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f1daa-bc15-4ca5-9272-4b7dbc9115de admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:48.263669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef69025-d884-4670-905d-76d4d69c0d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:48.264516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef69025-d884-4670-905d-76d4d69c0d16 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:48.264868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef69025-d884-4670-905d-76d4d69c0d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:48.266619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef69025-d884-4670-905d-76d4d69c0d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:48.276608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef69025-d884-4670-905d-76d4d69c0d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:48.278392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef69025-d884-4670-905d-76d4d69c0d16 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:48.289315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1136/2244] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:39:48.777244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:39:48.778853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:39:48.791342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a06c6-d786-4f2f-a9ef-8db75c905b09 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:39:48.796073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:39:48.796908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:39:48.797201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:39:48.797482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] PATH_INFO: `/v3/projects/7a82efb520f34fe684bffa4a8692180d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:39:48.799282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a82efb520f34fe684bffa4a8692180d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:39:48.800725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:39:48.937167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a82efb520f34fe684bffa4a8692180d', 'request_id': 'req-bbdc017b-00db-423b-a385-ff41ea276866'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:49.022801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a82efb520f34fe684bffa4a8692180d', 'request_id': 'req-bbdc017b-00db-423b-a385-ff41ea276866'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:49.029027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a82efb520f34fe684bffa4a8692180d', 'request_id': 'req-bbdc017b-00db-423b-a385-ff41ea276866'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:49.041101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] The token cache is being invalidate because project 7a82efb520f34fe684bffa4a8692180d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:39:49.041588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:49.042303 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bbdc017b-00db-423b-a385-ff41ea276866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:39:49.043880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1109/2245] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:39:48 2022] DELETE /identity/v3/projects/7a82efb520f34fe684bffa4a8692180d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:39:58.285196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:39:59.038015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:40:03.253534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:03.254711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:03.255288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:03.279701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:03.281173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:03.324948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:03.325507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:03.325851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:03.346972 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:03.348497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:03.360440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1137/2246] 10.222.0.128 () {58 vars in 972 bytes} [Tue Jul 26 15:40:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:40:03.376163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:03.378248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:03.421499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1110/2247] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:40:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:40:03.434604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:03.436772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:03.486816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923b402-c074-4bcd-a16b-d703b6300dc3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:03.492562 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c0b29-2d25-40f7-9fdf-a2bc4acfd38f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:03.495557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c0b29-2d25-40f7-9fdf-a2bc4acfd38f service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:03.495557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c0b29-2d25-40f7-9fdf-a2bc4acfd38f service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:03.495557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c0b29-2d25-40f7-9fdf-a2bc4acfd38f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:03.504272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c0b29-2d25-40f7-9fdf-a2bc4acfd38f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:03.505922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c0b29-2d25-40f7-9fdf-a2bc4acfd38f service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:03.518351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1138/2248] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:40:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:40:03.551357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f9833-32c4-44c5-b432-3ffac7a9b4f2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:03.557847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:03.558709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:03.559093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:03.559543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:03.572172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:03.574463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:03.592982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1111/2249] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:40:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:40:04.550607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45fb2-ab5e-4da0-b644-34d59a0ccfbb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:04.551055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45fb2-ab5e-4da0-b644-34d59a0ccfbb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:04.551174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45fb2-ab5e-4da0-b644-34d59a0ccfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:04.571523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bae45fb2-ab5e-4da0-b644-34d59a0ccfbb None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:04.573460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bae45fb2-ab5e-4da0-b644-34d59a0ccfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:04.581324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1139/2250] 10.222.0.128 () {58 vars in 972 bytes} [Tue Jul 26 15:40:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:40:04.595633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:04.606525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48fe66-2bca-4f1a-a999-37192df8ff6c service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:04.611614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:04.612271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:04.612541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:04.612821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:04.621355 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:04.622944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:04.631173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1112/2251] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:40:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:40:05.317391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ba1f-09f1-4071-94f7-5c3821da53b7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:05.317682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ba1f-09f1-4071-94f7-5c3821da53b7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:05.317961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ba1f-09f1-4071-94f7-5c3821da53b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:05.339110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0236ba1f-09f1-4071-94f7-5c3821da53b7 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:05.340337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0236ba1f-09f1-4071-94f7-5c3821da53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:05.347604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1140/2252] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:40:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:40:05.360193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:05.373481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed6a8-e10b-46b3-93d9-9cc3ef0186a7 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:05.378787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:05.379555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:05.379731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:05.380023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:05.389074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:05.390699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:05.399745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1113/2253] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:40:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:40:06.003396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe099311-66e9-4fba-98a0-538e6ff7a2a7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:06.003923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe099311-66e9-4fba-98a0-538e6ff7a2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:06.004274 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe099311-66e9-4fba-98a0-538e6ff7a2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:06.028856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fe099311-66e9-4fba-98a0-538e6ff7a2a7 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:06.032143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe099311-66e9-4fba-98a0-538e6ff7a2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:06.038403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1141/2254] 10.222.0.128 () {58 vars in 972 bytes} [Tue Jul 26 15:40:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:40:06.075710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:06.096852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac93f37-4fd0-4dd3-ac59-d92cb2df79d5 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:06.109874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:06.111255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:06.111891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:06.112379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:06.139843 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:06.141486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:06.151818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1114/2255] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:40:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:40:07.391189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:07.392040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:07.392573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:07.414137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] MFA Rules not processed for user `c2bf7ea384a142d1bc23b23ab503fca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:07.415441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:07.508236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1142/2256] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:40:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:40:07.876497 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:07.878896 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:07.990424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdb6ee-65c4-4034-bbe4-21943492050e service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:07.995266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29b17e-f43a-4c9b-9f48-ccd99ddfbdaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:07.996017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b17e-f43a-4c9b-9f48-ccd99ddfbdaa service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:07.996403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b17e-f43a-4c9b-9f48-ccd99ddfbdaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:07.996808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b17e-f43a-4c9b-9f48-ccd99ddfbdaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:08.003994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3f29b17e-f43a-4c9b-9f48-ccd99ddfbdaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:08.017202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:40:08.126199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29b17e-f43a-4c9b-9f48-ccd99ddfbdaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:08.127929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29b17e-f43a-4c9b-9f48-ccd99ddfbdaa service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:08.153047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1115/2257] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:40:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:40:09.388889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:09.390432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:09.455282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:40:09.553393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5d8b5-9e79-43cb-b94a-264cb627b938 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:09.560864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:09.561532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:09.561806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:09.562254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] PATH_INFO: `/v3/users/c746449a143140fc87ee685c4fc5f3ea` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:09.564067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c746449a143140fc87ee685c4fc5f3ea)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:09.566770 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:09.761154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c746449a143140fc87ee685c4fc5f3ea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:09.762236 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c746449a143140fc87ee685c4fc5f3ea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:09.773699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c746449a143140fc87ee685c4fc5f3ea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:09.789147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c746449a143140fc87ee685c4fc5f3ea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:09.833140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c746449a143140fc87ee685c4fc5f3ea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:09.841171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1143/2258] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:40:09 2022] DELETE /identity/v3/users/c746449a143140fc87ee685c4fc5f3ea => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:40:09.843952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750746a2-8f3a-4788-8e85-b5c2d57f8636 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:09.844505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750746a2-8f3a-4788-8e85-b5c2d57f8636 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:09.844680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750746a2-8f3a-4788-8e85-b5c2d57f8636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:09.853922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:09.855844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:09.867489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-750746a2-8f3a-4788-8e85-b5c2d57f8636 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:09.869072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-750746a2-8f3a-4788-8e85-b5c2d57f8636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:09.951850 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1116/2259] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:40:09.959753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a896ed-f02c-4532-8156-1d41756a7664 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:09.964557 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750746a2-8f3a-4788-8e85-b5c2d57f8636 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:09.970084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2c79e-478b-460c-96bc-5c94bde3f325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:09.970084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2c79e-478b-460c-96bc-5c94bde3f325 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:09.970084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2c79e-478b-460c-96bc-5c94bde3f325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:09.970084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2c79e-478b-460c-96bc-5c94bde3f325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:09.972435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b1f2c79e-478b-460c-96bc-5c94bde3f325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:09.978453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750746a2-8f3a-4788-8e85-b5c2d57f8636 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:09.983226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:09.983956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:09.984352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:09.984780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:10.005001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:10.007032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:10.031542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1117/2260] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:40:10.039585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:40:10.072629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2c79e-478b-460c-96bc-5c94bde3f325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:10.074750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2c79e-478b-460c-96bc-5c94bde3f325 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:10.088670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1144/2261] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:40:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:40:10.284632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:10.286478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:10.300692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ba164-ef9b-4b1f-a44f-aef5af5101ca service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:10.305351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:10.305998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:10.306286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:10.306574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] PATH_INFO: `/v3/projects/d432a53d4fc44e8e9e557f7e3bca2c47` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:10.308256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d432a53d4fc44e8e9e557f7e3bca2c47)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:10.310517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:10.422134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd432a53d4fc44e8e9e557f7e3bca2c47', 'request_id': 'req-f3e60768-6ac4-41bc-b8fa-678c44f5a433'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:10.493451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd432a53d4fc44e8e9e557f7e3bca2c47', 'request_id': 'req-f3e60768-6ac4-41bc-b8fa-678c44f5a433'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:10.500180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd432a53d4fc44e8e9e557f7e3bca2c47', 'request_id': 'req-f3e60768-6ac4-41bc-b8fa-678c44f5a433'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:10.512538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] The token cache is being invalidate because project d432a53d4fc44e8e9e557f7e3bca2c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:40:10.512834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:10.513485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:40:10.514984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1118/2262] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:40:10 2022] DELETE /identity/v3/projects/d432a53d4fc44e8e9e557f7e3bca2c47 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:40:14.626313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb9553-7d94-4554-bdcd-2899f7164039 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:14.626929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb9553-7d94-4554-bdcd-2899f7164039 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:14.626929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb9553-7d94-4554-bdcd-2899f7164039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:14.652115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9cbb9553-7d94-4554-bdcd-2899f7164039 None None] MFA Rules not processed for user `494aa7af44284866b02f2a295b8b05aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:40:14.653985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9cbb9553-7d94-4554-bdcd-2899f7164039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:14.741574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1145/2263] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:40:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:40:16.383475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:16.386763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:16.398999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:40:16.456145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60768-6ac4-41bc-b8fa-678c44f5a433 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:16.465928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:16.466645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:16.467003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:16.467635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:16.469484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:16.550948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:16.554003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:16.565638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1119/2264] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:40:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:40:22.660726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb9553-7d94-4554-bdcd-2899f7164039 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:22.671131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb9553-7d94-4554-bdcd-2899f7164039 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:22.674920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:22.677986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:22.677986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:22.677986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:22.677986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:22.754486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:22.758154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:22.768518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1146/2265] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:40:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:40:25.037676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:25.039263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:25.122249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ff9eb-91d8-49f8-8f24-86f417ec00c1 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:25.128266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:25.128949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:25.129382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:25.129675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:25.131599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:25.145306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:25.147501 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:25.160158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1120/2266] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:40:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:40:32.763416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=64 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:40:35.155177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:40:59.259731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:40:59.261290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:59.277430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bbe1-b43f-4545-9f52-1cd88b25f9f1 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:40:59.283464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:40:59.284089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:40:59.284403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:40:59.284710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:40:59.287016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:40:59.345182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:40:59.346784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:40:59.356995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1147/2267] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:00.494787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:00.496209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:00.508740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e14c-7a5d-4824-ba8c-0fcd4a96a581 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:00.512351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:00.512994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:00.513276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:00.513548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:00.515447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:00.526472 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:00.527902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:00.537076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1121/2268] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:05.122564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:05.124551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:05.184560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e10d3-ed25-446d-a2e0-234e32e1af3f service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:05.189053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755379f-c741-4a2c-824c-d744fc2e1e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:05.189640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755379f-c741-4a2c-824c-d744fc2e1e81 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:05.189925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755379f-c741-4a2c-824c-d744fc2e1e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:05.190208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755379f-c741-4a2c-824c-d744fc2e1e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:05.192024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8755379f-c741-4a2c-824c-d744fc2e1e81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:05.257730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755379f-c741-4a2c-824c-d744fc2e1e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:05.259317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755379f-c741-4a2c-824c-d744fc2e1e81 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:05.269741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1148/2269] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:06.541751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:06.552266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016774c9-eebe-44f5-b360-2d5d31ae9528 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:06.558384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:06.560449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:06.560449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:06.560449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:06.561634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:06.575028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:06.576810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:06.589775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1122/2270] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:11.592683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:11.593180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:11.593323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:11.612609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] MFA Rules not processed for user `e2e9553297c842b3a3c308cdd09aad8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:11.613817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:11.621260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1149/2271] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:11.936741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:11.946879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787e8d7-fe0a-4337-a40f-db6570dab603 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:11.950450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:11.951060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:11.951346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:11.951626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:11.953414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:11.964525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:11.965983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:11.974704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1123/2272] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:12.373679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:12.375071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:12.427303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:12.448688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c2738-1f94-44b9-b564-8c83b752b5f4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:12.453702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:12.455514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:12.455514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:12.455514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:12.456737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:12.469404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:12.471087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:12.482119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1150/2273] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:19.483538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:19.495201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3cab6-6df1-4feb-8072-7ecca08007a0 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:19.499082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:19.499742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:19.500029 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:19.500310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:19.502112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:19.564418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:19.566047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:19.575736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1124/2274] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:22.478153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=11 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:22.479606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:22.492769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780775b9-28c8-4201-b553-fd4a3a3ac3af service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:22.498702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb2637-edf6-4b6b-b1d9-501d49e96ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:22.499413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2637-edf6-4b6b-b1d9-501d49e96ab3 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:22.499760 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2637-edf6-4b6b-b1d9-501d49e96ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:22.502949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2637-edf6-4b6b-b1d9-501d49e96ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:22.502949 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2fbb2637-edf6-4b6b-b1d9-501d49e96ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:22.515164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb2637-edf6-4b6b-b1d9-501d49e96ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:22.516796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb2637-edf6-4b6b-b1d9-501d49e96ab3 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:22.529849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1151/2275] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:23.345809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:23.372542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebcd4c-a961-49f0-b793-13817d246555 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:23.380512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:23.381435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:23.382333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:23.382841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:23.385530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:23.402666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:23.405057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:23.421993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1125/2276] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:30.822692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f0800-f814-439b-9061-9288eeeb16e7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:30.823278 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f0800-f814-439b-9061-9288eeeb16e7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:30.823378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f0800-f814-439b-9061-9288eeeb16e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:30.843689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3e5f0800-f814-439b-9061-9288eeeb16e7 None None] MFA Rules not processed for user `c2e3c15511fe433a9278248c528c86e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:30.844982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e5f0800-f814-439b-9061-9288eeeb16e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:30.853448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1152/2277] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:31.172885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:31.177287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=53 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:31.184079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646965f-6093-4b67-83ad-0053a78cdd52 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:31.187678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:31.188283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:31.188592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:31.189038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:31.190797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:31.202219 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:31.203880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:31.213326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1126/2278] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:31.438920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f0800-f814-439b-9061-9288eeeb16e7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:31.450895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f0800-f814-439b-9061-9288eeeb16e7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:31.455155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:31.455906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:31.456377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:31.456791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:31.462195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:31.539567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:31.541514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:31.552631 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1153/2279] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:35.805274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:35.815844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:35.837558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de37b9a-95f2-4a2a-ae17-23f92a9c4113 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:35.850149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:35.850149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:35.850149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:35.850149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:35.866584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:35.885929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:35.887705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:35.903121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1127/2280] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:36.034806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:36.036231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:36.060356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76948937-bd07-47f4-b091-9377096c2f45 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:36.068415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:36.068415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:36.071566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:36.072155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] PATH_INFO: `/v3/users/c2bf7ea384a142d1bc23b23ab503fca0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:36.075249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2bf7ea384a142d1bc23b23ab503fca0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:36.077529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:36.287015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2bf7ea384a142d1bc23b23ab503fca0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.289048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2bf7ea384a142d1bc23b23ab503fca0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.296433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2bf7ea384a142d1bc23b23ab503fca0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.310216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2bf7ea384a142d1bc23b23ab503fca0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.395933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-417bbab8-1221-4e31-8b5c-d2556a08c492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2bf7ea384a142d1bc23b23ab503fca0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.402637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1154/2281] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:41:36 2022] DELETE /identity/v3/users/c2bf7ea384a142d1bc23b23ab503fca0 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:36.671524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:36.684211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255302bf-d095-465f-af12-ab69f7c89e97 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:36.757169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:36.757856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:36.758143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:36.758548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] PATH_INFO: `/v3/projects/cc117969e9394769a471d512ce80734a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:36.759715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc117969e9394769a471d512ce80734a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:36.761787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:36.880876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc117969e9394769a471d512ce80734a', 'request_id': 'req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.955856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc117969e9394769a471d512ce80734a', 'request_id': 'req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.963579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc117969e9394769a471d512ce80734a', 'request_id': 'req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.976146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] The token cache is being invalidate because project cc117969e9394769a471d512ce80734a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:41:36.976651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.977390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df1f4de8-4ad4-452f-a6f3-4e7e3b83fa23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:36.982285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1128/2282] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:41:36 2022] DELETE /identity/v3/projects/cc117969e9394769a471d512ce80734a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:36.984245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b321ec-4e73-4e72-8214-99ef8e0726a4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:36.984589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b321ec-4e73-4e72-8214-99ef8e0726a4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:36.984862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b321ec-4e73-4e72-8214-99ef8e0726a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.009106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a8b321ec-4e73-4e72-8214-99ef8e0726a4 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:37.010565 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a8b321ec-4e73-4e72-8214-99ef8e0726a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:37.056016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:37.074282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1155/2283] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:37.082779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d8209-8712-4e3c-9379-45ffa9372e06 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.083182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d8209-8712-4e3c-9379-45ffa9372e06 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.083539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d8209-8712-4e3c-9379-45ffa9372e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.105188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-759d8209-8712-4e3c-9379-45ffa9372e06 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:37.106494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-759d8209-8712-4e3c-9379-45ffa9372e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:37.115164 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1129/2284] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:41:37.119476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b321ec-4e73-4e72-8214-99ef8e0726a4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.132834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b321ec-4e73-4e72-8214-99ef8e0726a4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.138050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.138905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.139610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.140065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.140819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:41:37.141135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.142786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:37.156345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1156/2285] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:41:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:41:37.160682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d8209-8712-4e3c-9379-45ffa9372e06 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.171844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d8209-8712-4e3c-9379-45ffa9372e06 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.175321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94740560-1062-447c-9b26-964781930396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.175928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94740560-1062-447c-9b26-964781930396 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.176207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94740560-1062-447c-9b26-964781930396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.176529 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94740560-1062-447c-9b26-964781930396 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.177198 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94740560-1062-447c-9b26-964781930396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.179103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94740560-1062-447c-9b26-964781930396 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:37.301517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1130/2286] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:41:37 2022] POST /identity/v3/projects => generated 368 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:37.306198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.316232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2a12-4454-4524-8056-c13385d680cf admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.372891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.373515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.373806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.374087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.374761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.376727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:37.442344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1157/2287] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:41:37 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:37.447138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94740560-1062-447c-9b26-964781930396 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.458494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94740560-1062-447c-9b26-964781930396 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.464950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.465587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.465878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.466156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.466863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.468493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:37.478938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1131/2288] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:37 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:37.482977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.493285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544a917-2632-4f14-8c74-d80a6642b9a2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.498191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3369d9-a1e3-49ec-849a-1879e743dc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.498831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3369d9-a1e3-49ec-849a-1879e743dc09 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.499115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3369d9-a1e3-49ec-849a-1879e743dc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.499393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3369d9-a1e3-49ec-849a-1879e743dc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.500100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3369d9-a1e3-49ec-849a-1879e743dc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.501741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3369d9-a1e3-49ec-849a-1879e743dc09 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:37.513174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1158/2289] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:37 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:37.517413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.529016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa357ce-f69f-4550-8fdf-639d00da3848 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.533649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.534302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.534588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.534886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] PATH_INFO: `/v3/projects/d49020e611e640f89179e24fac70c028/users/113c3cbd67b0496796d724e3c4a1fb4d/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.550894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49020e611e640f89179e24fac70c028, user_id=113c3cbd67b0496796d724e3c4a1fb4d, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.553867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:37.624874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1132/2290] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:37 2022] PUT /identity/v3/projects/d49020e611e640f89179e24fac70c028/users/113c3cbd67b0496796d724e3c4a1fb4d/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:41:37.638830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42c59-09c5-495e-8534-8e82bd67732a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.639186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42c59-09c5-495e-8534-8e82bd67732a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.639513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42c59-09c5-495e-8534-8e82bd67732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.668382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3cf42c59-09c5-495e-8534-8e82bd67732a None None] MFA Rules not processed for user `113c3cbd67b0496796d724e3c4a1fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:37.669659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3cf42c59-09c5-495e-8534-8e82bd67732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:37.734698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1159/2291] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:37.739362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.744332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:37.804116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291fe92-31a7-4645-980d-e74a34927502 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.809013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.812413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.812413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.812413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.812413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.812848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:37.911595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1133/2292] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:41:37 2022] POST /identity/v3/projects => generated 370 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:37.916371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf42c59-09c5-495e-8534-8e82bd67732a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:37.927152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf42c59-09c5-495e-8534-8e82bd67732a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:37.972992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:37.989481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:37.990042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:37.990333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:37.991193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:37.992327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:37.993866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:38.060930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1160/2293] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:41:37 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:38.064726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:38.075511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835025be-87b2-4132-96e7-4d91b8486975 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:38.080203 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:38.080854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:38.081144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:38.081422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:38.082127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:38.083796 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:38.101851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1134/2294] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:38 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:38.106029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:38.116868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81203e-ea23-4d6c-be30-7adf273ab77d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:38.121942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:38.122699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:38.122935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:38.123288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] PATH_INFO: `/v3/projects/85dd122f2a344b7a8786d93020f40aac/users/1f462d1aefd84c6d91a0731f90060d81/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:38.141140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] RBAC: Authorizing `identity:create_grant(project_id=85dd122f2a344b7a8786d93020f40aac, user_id=1f462d1aefd84c6d91a0731f90060d81, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:38.143934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:38.214493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1161/2295] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:38 2022] PUT /identity/v3/projects/85dd122f2a344b7a8786d93020f40aac/users/1f462d1aefd84c6d91a0731f90060d81/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:38.218698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:38.229719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc43a8-a664-44b8-8c41-0d3379df9533 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:38.291326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626cdc4-999a-4589-a3d6-ac7b4da6f3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:38.291990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626cdc4-999a-4589-a3d6-ac7b4da6f3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:38.292273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626cdc4-999a-4589-a3d6-ac7b4da6f3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:38.292594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626cdc4-999a-4589-a3d6-ac7b4da6f3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:38.293304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626cdc4-999a-4589-a3d6-ac7b4da6f3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:38.294921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626cdc4-999a-4589-a3d6-ac7b4da6f3b0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:38.306852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1135/2296] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:38 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:38.311381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:38.323947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03703c6-5246-4e5e-a5c6-4630207d321a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:38.328604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:38.329277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:38.329588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:38.329864 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] PATH_INFO: `/v3/projects/85dd122f2a344b7a8786d93020f40aac/users/1f462d1aefd84c6d91a0731f90060d81/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:38.331804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85dd122f2a344b7a8786d93020f40aac, user_id=1f462d1aefd84c6d91a0731f90060d81, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:38.334544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:38.345482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1162/2297] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:38 2022] PUT /identity/v3/projects/85dd122f2a344b7a8786d93020f40aac/users/1f462d1aefd84c6d91a0731f90060d81/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:38.359524 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba9ab1-c63c-43db-826d-5e4e0670bca4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:38.359819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba9ab1-c63c-43db-826d-5e4e0670bca4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:38.360126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba9ab1-c63c-43db-826d-5e4e0670bca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:38.389637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5dba9ab1-c63c-43db-826d-5e4e0670bca4 None None] MFA Rules not processed for user `1f462d1aefd84c6d91a0731f90060d81`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:38.390878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5dba9ab1-c63c-43db-826d-5e4e0670bca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:38.458772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:38.465454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1136/2298] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:38.480554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:38.482342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:38.542291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27908d51-706e-4c57-b4cb-622112f288b5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:38.545891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab9ea0-996e-41f2-8eb9-44e1e7b3dd13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:38.546512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9ea0-996e-41f2-8eb9-44e1e7b3dd13 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:38.547184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9ea0-996e-41f2-8eb9-44e1e7b3dd13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:38.547184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9ea0-996e-41f2-8eb9-44e1e7b3dd13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:38.558962 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab9ea0-996e-41f2-8eb9-44e1e7b3dd13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:38.560433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab9ea0-996e-41f2-8eb9-44e1e7b3dd13 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:38.570834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1163/2299] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:41.423828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba9ab1-c63c-43db-826d-5e4e0670bca4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:41.486960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba9ab1-c63c-43db-826d-5e4e0670bca4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:41.492156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:41.492845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:41.493121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:41.493404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] PATH_INFO: `/v3/users/113c3cbd67b0496796d724e3c4a1fb4d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:41.494596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] RBAC: Authorizing `identity:delete_user(user_id=113c3cbd67b0496796d724e3c4a1fb4d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:41.496561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:41.758485 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113c3cbd67b0496796d724e3c4a1fb4d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:41.767583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113c3cbd67b0496796d724e3c4a1fb4d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:41.768334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113c3cbd67b0496796d724e3c4a1fb4d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:41.864022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113c3cbd67b0496796d724e3c4a1fb4d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:41.864764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113c3cbd67b0496796d724e3c4a1fb4d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:41.883562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1137/2300] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:41:41 2022] DELETE /identity/v3/users/113c3cbd67b0496796d724e3c4a1fb4d => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:41.888256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:41.888751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:41.895826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:41.908739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:41.909851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:41.971379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1164/2301] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:41.982037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:41.985753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:42.047888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a67e6-ebf1-4e18-9119-c5341ace638a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:42.051379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:42.051992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:42.052271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:42.052615 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:42.062079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:42.063549 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:42.073558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1138/2302] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:42.574923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:42.576423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:42.591647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7dd6b-33df-4bf7-a268-5f13a442ed2f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:42.596699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:42.597527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:42.598003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:42.598303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] PATH_INFO: `/v3/projects/d49020e611e640f89179e24fac70c028` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:42.599564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49020e611e640f89179e24fac70c028)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:42.602163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:42.728056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:42.740093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49020e611e640f89179e24fac70c028', 'request_id': 'req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:42.746986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49020e611e640f89179e24fac70c028', 'request_id': 'req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:42.753575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49020e611e640f89179e24fac70c028', 'request_id': 'req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:42.812365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] The token cache is being invalidate because project d49020e611e640f89179e24fac70c028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:41:42.812696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:42.813349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:42.814912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1165/2303] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:41:42 2022] DELETE /identity/v3/projects/d49020e611e640f89179e24fac70c028 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:42.819506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:42.830038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bda730-b722-48e8-97e8-f49589834d23 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:42.850945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:42.893282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:42.894068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:42.894363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:42.894648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] PATH_INFO: `/v3/users/1f462d1aefd84c6d91a0731f90060d81` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:42.895867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f462d1aefd84c6d91a0731f90060d81)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:42.897847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:43.077852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f462d1aefd84c6d91a0731f90060d81'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.084979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f462d1aefd84c6d91a0731f90060d81'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.088294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f462d1aefd84c6d91a0731f90060d81'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.143817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f462d1aefd84c6d91a0731f90060d81'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.144688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-277f5406-a4a9-4875-9f07-59d407cec224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f462d1aefd84c6d91a0731f90060d81'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.163214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1139/2304] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:41:42 2022] DELETE /identity/v3/users/1f462d1aefd84c6d91a0731f90060d81 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:43.578319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:43.579990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:43.593907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63ee59-cfa0-41ca-9240-1cfecc30bd17 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:43.655632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:43.656291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:43.656605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:43.656885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] PATH_INFO: `/v3/projects/85dd122f2a344b7a8786d93020f40aac` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:43.658066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85dd122f2a344b7a8786d93020f40aac)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:43.660034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:43.749272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85dd122f2a344b7a8786d93020f40aac', 'request_id': 'req-1e8bb39f-0228-4161-b2d8-bd9b4f937135'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.757768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85dd122f2a344b7a8786d93020f40aac', 'request_id': 'req-1e8bb39f-0228-4161-b2d8-bd9b4f937135'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.764321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85dd122f2a344b7a8786d93020f40aac', 'request_id': 'req-1e8bb39f-0228-4161-b2d8-bd9b4f937135'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.829655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] The token cache is being invalidate because project 85dd122f2a344b7a8786d93020f40aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:41:43.829946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.836417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1e8bb39f-0228-4161-b2d8-bd9b4f937135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:43.836417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1166/2305] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:41:43 2022] DELETE /identity/v3/projects/85dd122f2a344b7a8786d93020f40aac => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:43.837060 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19283c8-1aed-477c-876a-dbb9e771616c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:43.837577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19283c8-1aed-477c-876a-dbb9e771616c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:43.837677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19283c8-1aed-477c-876a-dbb9e771616c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:43.858128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f19283c8-1aed-477c-876a-dbb9e771616c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:43.859442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f19283c8-1aed-477c-876a-dbb9e771616c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:43.926223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1140/2306] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:43.937103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ae65-218e-4151-a63d-eb861db9a95b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:43.937600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ae65-218e-4151-a63d-eb861db9a95b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:43.937724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ae65-218e-4151-a63d-eb861db9a95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:43.958502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-96b2ae65-218e-4151-a63d-eb861db9a95b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:43.959789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96b2ae65-218e-4151-a63d-eb861db9a95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:43.969777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1167/2307] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:43.974154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19283c8-1aed-477c-876a-dbb9e771616c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:43.986564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19283c8-1aed-477c-876a-dbb9e771616c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:43.995474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:43.996244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.009277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.009277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.009277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:41:44.009277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.009277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:44.016369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1141/2308] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:41:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:41:44.021835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2ae65-218e-4151-a63d-eb861db9a95b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:44.033430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2ae65-218e-4151-a63d-eb861db9a95b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:44.040722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:44.041838 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.042122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.042395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.043074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.045130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:44.146849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1168/2309] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:41:44 2022] POST /identity/v3/projects => generated 370 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:44.153496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:44.166184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9c56c-1075-48b4-9ba7-5df73f658598 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:44.197615 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:44.231492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:44.232117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.232400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.232707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.233395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.235360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:44.333205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1142/2310] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:41:44 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:44.338229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:44.348614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938cbd1-64d0-459c-ba68-2f08ecd1c585 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:44.353584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:44.354219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.354507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.354788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.355500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.357149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:44.370869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1169/2311] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:44 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:41:44.372429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:44.383439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8d7da-f82d-4f62-b642-6681d8e1b645 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:44.389013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206862f-cca4-4b22-a85b-ae6012ab684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:44.389674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206862f-cca4-4b22-a85b-ae6012ab684c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.389964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206862f-cca4-4b22-a85b-ae6012ab684c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.390246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206862f-cca4-4b22-a85b-ae6012ab684c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.390956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206862f-cca4-4b22-a85b-ae6012ab684c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.392589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206862f-cca4-4b22-a85b-ae6012ab684c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:44.406764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1143/2312] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:44 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:44.407922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:44.418134 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379dddc-6e5c-4b21-83c1-47808234b5d7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:44.423119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:44.423807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.424096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.424381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] PATH_INFO: `/v3/projects/21766e7ea27e4f6fb03aacfdebcc9458/users/878e4aeda41c4d1f846534d99c61e859/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.434507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:44.441076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21766e7ea27e4f6fb03aacfdebcc9458, user_id=878e4aeda41c4d1f846534d99c61e859, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.443829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:44.518522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1170/2313] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:44 2022] PUT /identity/v3/projects/21766e7ea27e4f6fb03aacfdebcc9458/users/878e4aeda41c4d1f846534d99c61e859/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:44.530874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb01a2-d38f-4cfe-ab09-cbb526414cfd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.531513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb01a2-d38f-4cfe-ab09-cbb526414cfd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.531911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb01a2-d38f-4cfe-ab09-cbb526414cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.566619 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dcbb01a2-d38f-4cfe-ab09-cbb526414cfd None None] MFA Rules not processed for user `878e4aeda41c4d1f846534d99c61e859`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:44.568010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dcbb01a2-d38f-4cfe-ab09-cbb526414cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:44.668290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1144/2314] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:41:44.672539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:44.758335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee43edd-04a1-417a-9c84-d6eec3b921b6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:44.764762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:44.765636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.765912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.766173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.766894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.768984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:44.880149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1171/2315] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:41:44 2022] POST /identity/v3/projects => generated 366 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:41:44.885587 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb01a2-d38f-4cfe-ab09-cbb526414cfd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:44.899959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb01a2-d38f-4cfe-ab09-cbb526414cfd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:44.976509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:44.977082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:44.977507 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:44.977914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:44.978860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:44.980843 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:45.068705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1145/2316] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:41:44 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:45.069918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:45.081332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b0bd0-ad8e-4ea7-b54d-7b9deca72710 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:45.085913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:45.086534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:45.087119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:45.087401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:45.088108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:45.089754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:45.112766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:45.113591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1172/2317] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:45 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:41:45.129436 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e73be0-fc0d-4901-b540-e4e485245bf6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:45.136875 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:45.137800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:45.138262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:45.139280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] PATH_INFO: `/v3/projects/4463fd78786f4f89880b70cb2b802178/users/bd7dfe9e03f348e68e4a58a618407973/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:45.161281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4463fd78786f4f89880b70cb2b802178, user_id=bd7dfe9e03f348e68e4a58a618407973, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:45.164714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:45.238531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:45.245496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1146/2318] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:45 2022] PUT /identity/v3/projects/4463fd78786f4f89880b70cb2b802178/users/bd7dfe9e03f348e68e4a58a618407973/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:41:45.249862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:45.265317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7774e-4a0c-4849-9590-3f5786d7ab65 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:45.405502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d39e81-42f5-444b-88b9-2c3becdeb7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:45.406411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39e81-42f5-444b-88b9-2c3becdeb7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:45.406787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39e81-42f5-444b-88b9-2c3becdeb7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:45.407898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39e81-42f5-444b-88b9-2c3becdeb7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:45.408762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d39e81-42f5-444b-88b9-2c3becdeb7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:45.410495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d39e81-42f5-444b-88b9-2c3becdeb7bd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:45.437061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1173/2319] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:45 2022] GET /identity/v3/roles => generated 1427 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:45.438560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:45.454332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47430ce6-65e1-4cd5-b4ec-b59e1e650567 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:45.470583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:45.473139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:45.473139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:45.473139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] PATH_INFO: `/v3/projects/4463fd78786f4f89880b70cb2b802178/users/bd7dfe9e03f348e68e4a58a618407973/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:45.476563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4463fd78786f4f89880b70cb2b802178, user_id=bd7dfe9e03f348e68e4a58a618407973, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:45.480451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:45.520673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1147/2320] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:45 2022] PUT /identity/v3/projects/4463fd78786f4f89880b70cb2b802178/users/bd7dfe9e03f348e68e4a58a618407973/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:45.525098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:45.526321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:45.526694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:45.588633 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] MFA Rules not processed for user `bd7dfe9e03f348e68e4a58a618407973`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:45.593688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:45.658203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:45.733285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1174/2321] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:45.742207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:45.744484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:45.887599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d250-5f3d-45b1-bd4e-97ddcc15f4eb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:45.893595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880e2b0-711c-4a51-a703-78fa46469169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:45.897139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880e2b0-711c-4a51-a703-78fa46469169 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:45.897139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880e2b0-711c-4a51-a703-78fa46469169 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:45.897139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880e2b0-711c-4a51-a703-78fa46469169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:45.919102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880e2b0-711c-4a51-a703-78fa46469169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:45.925615 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880e2b0-711c-4a51-a703-78fa46469169 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:45.943812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1148/2322] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:46.148671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:46.151316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:46.169810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b042bc-0ab7-4479-8af8-ec145b49a6c1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:46.175959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:46.176672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:46.177050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:46.177325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:46.256859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:46.259117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:46.271312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1175/2323] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:51.500229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9b34b-34b7-4e1e-b32b-577cc80815b7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:51.500770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9b34b-34b7-4e1e-b32b-577cc80815b7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:51.500868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9b34b-34b7-4e1e-b32b-577cc80815b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:51.521173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-15f9b34b-34b7-4e1e-b32b-577cc80815b7 None None] MFA Rules not processed for user `494aa7af44284866b02f2a295b8b05aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:51.522416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15f9b34b-34b7-4e1e-b32b-577cc80815b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:51.571477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:51.588886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1149/2324] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:51.654998 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c0488-cf8a-4550-a5f9-032be82aceb1 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:51.659891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:51.660585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:51.660878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:51.661148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] PATH_INFO: `/v3/users/878e4aeda41c4d1f846534d99c61e859` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:51.662367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=878e4aeda41c4d1f846534d99c61e859)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:51.665103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:51.908805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878e4aeda41c4d1f846534d99c61e859'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:51.909459 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878e4aeda41c4d1f846534d99c61e859'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:51.917367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878e4aeda41c4d1f846534d99c61e859'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:51.935176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878e4aeda41c4d1f846534d99c61e859'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:52.029810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878e4aeda41c4d1f846534d99c61e859'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:52.033184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:52.040928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1176/2325] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:41:51 2022] DELETE /identity/v3/users/878e4aeda41c4d1f846534d99c61e859 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:41:52.045721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:52.046026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:52.046363 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:52.066393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:52.067746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:52.138510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1150/2326] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:52.151746 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:52.153308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:52.221904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea8c35-75da-4d9a-9241-af8b9be3b468 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:52.225610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:52.226267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:52.226553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:52.226867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:52.236756 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:52.238366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:52.249143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1177/2327] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:52.679382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:52.682126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:52.700588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4737e-14c2-4c53-b4f2-92c90dc788e7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:52.710001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:52.710770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:52.710955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:52.711335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] PATH_INFO: `/v3/projects/21766e7ea27e4f6fb03aacfdebcc9458` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:52.713101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21766e7ea27e4f6fb03aacfdebcc9458)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:52.714969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:52.817212 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:52.832170 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21766e7ea27e4f6fb03aacfdebcc9458', 'request_id': 'req-77e6806c-ce9d-4c97-a342-773ad088abf2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:52.943416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21766e7ea27e4f6fb03aacfdebcc9458', 'request_id': 'req-77e6806c-ce9d-4c97-a342-773ad088abf2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:52.949786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21766e7ea27e4f6fb03aacfdebcc9458', 'request_id': 'req-77e6806c-ce9d-4c97-a342-773ad088abf2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:52.963946 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] The token cache is being invalidate because project 21766e7ea27e4f6fb03aacfdebcc9458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:41:52.964231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:52.964955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:52.966480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1151/2328] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:41:52 2022] DELETE /identity/v3/projects/21766e7ea27e4f6fb03aacfdebcc9458 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:52.971291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:52.982600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ace76-00f5-4cef-a14d-cc349ad30e12 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:53.041485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:53.042143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:53.042422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:53.042698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] PATH_INFO: `/v3/users/bd7dfe9e03f348e68e4a58a618407973` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:53.043903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7dfe9e03f348e68e4a58a618407973)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:53.045888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:53.269079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7dfe9e03f348e68e4a58a618407973'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:53.269784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7dfe9e03f348e68e4a58a618407973'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:53.278930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7dfe9e03f348e68e4a58a618407973'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:53.293330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7dfe9e03f348e68e4a58a618407973'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:53.377891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e9bae149-60f4-4b9e-a79a-5ccf86a087f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7dfe9e03f348e68e4a58a618407973'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:53.385779 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1178/2329] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:41:52 2022] DELETE /identity/v3/users/bd7dfe9e03f348e68e4a58a618407973 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:53.785884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:53.791482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:53.814900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6806c-ce9d-4c97-a342-773ad088abf2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:53.888428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:53.889138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:53.889492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:53.889882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] PATH_INFO: `/v3/projects/4463fd78786f4f89880b70cb2b802178` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:53.891523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4463fd78786f4f89880b70cb2b802178)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:53.893720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:54.013841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4463fd78786f4f89880b70cb2b802178', 'request_id': 'req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:54.114686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4463fd78786f4f89880b70cb2b802178', 'request_id': 'req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:54.121456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4463fd78786f4f89880b70cb2b802178', 'request_id': 'req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:54.132826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] The token cache is being invalidate because project 4463fd78786f4f89880b70cb2b802178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:41:54.133829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:54.134609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-29a0282d-fa9f-43ba-b7e1-d6ab30217f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:41:54.141301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1152/2330] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:41:53 2022] DELETE /identity/v3/projects/4463fd78786f4f89880b70cb2b802178 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:54.141890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e932a-0bdf-4078-9feb-04cba4653976 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.142324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e932a-0bdf-4078-9feb-04cba4653976 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.142589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e932a-0bdf-4078-9feb-04cba4653976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.171537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-702e932a-0bdf-4078-9feb-04cba4653976 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:54.172984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-702e932a-0bdf-4078-9feb-04cba4653976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:54.195898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:54.238627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1179/2331] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:54.246665 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51c62-4242-4be2-ab33-c81eb1662f1c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.246961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51c62-4242-4be2-ab33-c81eb1662f1c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.247425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51c62-4242-4be2-ab33-c81eb1662f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.270502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-92b51c62-4242-4be2-ab33-c81eb1662f1c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:54.271727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-92b51c62-4242-4be2-ab33-c81eb1662f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:54.280753 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1153/2332] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:41:54.284999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e932a-0bdf-4078-9feb-04cba4653976 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:54.296131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e932a-0bdf-4078-9feb-04cba4653976 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:54.301084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:54.301787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.302070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.302351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.303097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:41:54.303683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:54.305628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:54.317409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1180/2333] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:41:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:41:54.322555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51c62-4242-4be2-ab33-c81eb1662f1c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:54.333751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51c62-4242-4be2-ab33-c81eb1662f1c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:54.339379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:54.340008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.340315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.340661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.341377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:54.343298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:54.423678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1154/2334] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:41:54 2022] POST /identity/v3/projects => generated 402 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:54.428093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:54.439423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6d1f-568b-4907-b32d-879812ef0aee admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:54.534964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:54.535660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.535942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.541478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.541478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:54.541478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:54.653621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1181/2335] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:41:54 2022] POST /identity/v3/users => generated 302 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:54.658026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:54.668814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddad5a-7a6b-4b32-9c75-07a14e9cbdd4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:54.673876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:54.674680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.675040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.675542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.676338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:54.678492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:54.690674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1155/2336] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:54 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:54.694657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:54.705928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c8e5-03ea-40ea-b196-8695d821fa9d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:54.717514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b9667-da2c-41e8-b7ef-bbe079a348a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:54.718273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9667-da2c-41e8-b7ef-bbe079a348a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.718671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9667-da2c-41e8-b7ef-bbe079a348a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.719088 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9667-da2c-41e8-b7ef-bbe079a348a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.719920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b9667-da2c-41e8-b7ef-bbe079a348a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:54.721670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b9667-da2c-41e8-b7ef-bbe079a348a7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:54.734706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1182/2337] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:54 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:54.740933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:54.752693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a607d-49b8-4594-ab7b-b2f03808d86f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:54.757394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:54.758709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.759296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.759589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] PATH_INFO: `/v3/projects/3096c2160750462cafacb887a1e0b6ac/users/765fc2bd1b3f4112a640a37389a296a3/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.762911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:54.775676 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3096c2160750462cafacb887a1e0b6ac, user_id=765fc2bd1b3f4112a640a37389a296a3, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:54.778726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:54.829780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1156/2338] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:54 2022] PUT /identity/v3/projects/3096c2160750462cafacb887a1e0b6ac/users/765fc2bd1b3f4112a640a37389a296a3/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:41:54.840534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213a226-0bff-481f-9dd8-0111d0bed90e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:54.840726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213a226-0bff-481f-9dd8-0111d0bed90e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:54.841001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213a226-0bff-481f-9dd8-0111d0bed90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:54.871448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4213a226-0bff-481f-9dd8-0111d0bed90e None None] MFA Rules not processed for user `765fc2bd1b3f4112a640a37389a296a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:54.872813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4213a226-0bff-481f-9dd8-0111d0bed90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:54.950809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1183/2339] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:54.955319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:55.023099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c87642-a669-428b-a9a5-56bd2ba64538 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:55.028291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:55.028958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.029265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.029540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.030235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:55.032168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:55.166063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1157/2340] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:41:54 2022] POST /identity/v3/projects => generated 402 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:55.170172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213a226-0bff-481f-9dd8-0111d0bed90e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:55.185797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213a226-0bff-481f-9dd8-0111d0bed90e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:55.203009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:55.253632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921055-3bce-4818-8860-393d453754bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:55.254260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921055-3bce-4818-8860-393d453754bc admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.254680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921055-3bce-4818-8860-393d453754bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.255024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921055-3bce-4818-8860-393d453754bc admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.255702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58921055-3bce-4818-8860-393d453754bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:55.257911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58921055-3bce-4818-8860-393d453754bc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:55.361863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1184/2341] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:41:55 2022] POST /identity/v3/users => generated 308 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:41:55.363323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:55.382074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeb6e5-4539-411f-9489-b7bc92a84c1d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:55.394625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:55.398206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.398206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.398206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.398206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:55.401645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:55.419477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1158/2342] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:55 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:55.423727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921055-3bce-4818-8860-393d453754bc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:55.437257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921055-3bce-4818-8860-393d453754bc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:55.442859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:55.443622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.444096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.444435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] PATH_INFO: `/v3/projects/fac474c79f0f4bc68da46e48455c938b/users/b51c3bb7385748bdb0468ba0c8833aa7/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.465940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac474c79f0f4bc68da46e48455c938b, user_id=b51c3bb7385748bdb0468ba0c8833aa7, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:55.468758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:55.566818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1185/2343] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:55 2022] PUT /identity/v3/projects/fac474c79f0f4bc68da46e48455c938b/users/b51c3bb7385748bdb0468ba0c8833aa7/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:55.571767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:55.582424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fea09-9c04-40ef-9de4-19b11df045aa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:55.647795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f8d88-d5a2-4ab5-bf06-b269058eaa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:55.648489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f8d88-d5a2-4ab5-bf06-b269058eaa1e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.649182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f8d88-d5a2-4ab5-bf06-b269058eaa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.649182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f8d88-d5a2-4ab5-bf06-b269058eaa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.649935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f8d88-d5a2-4ab5-bf06-b269058eaa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:55.651655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f8d88-d5a2-4ab5-bf06-b269058eaa1e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:55.661856 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1159/2344] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:41:55 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:41:55.666544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:55.677043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af6109-1cfa-4722-82e7-304504c4fa6c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:55.682823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:55.683596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.683947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.684292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] PATH_INFO: `/v3/projects/fac474c79f0f4bc68da46e48455c938b/users/b51c3bb7385748bdb0468ba0c8833aa7/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.686505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac474c79f0f4bc68da46e48455c938b, user_id=b51c3bb7385748bdb0468ba0c8833aa7, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:55.689372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:55.700805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1186/2345] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:41:55 2022] PUT /identity/v3/projects/fac474c79f0f4bc68da46e48455c938b/users/b51c3bb7385748bdb0468ba0c8833aa7/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:41:55.712582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46d680-b67a-4560-9290-1b2a6f4e75b9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.712879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46d680-b67a-4560-9290-1b2a6f4e75b9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.713505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46d680-b67a-4560-9290-1b2a6f4e75b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.754258 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4a46d680-b67a-4560-9290-1b2a6f4e75b9 None None] MFA Rules not processed for user `b51c3bb7385748bdb0468ba0c8833aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:55.755998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4a46d680-b67a-4560-9290-1b2a6f4e75b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:55.807582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:41:55.839427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1160/2346] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:41:55 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:55.851065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:55.854311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:55.919699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74c34-00d2-458d-ad46-52a4a4b0494f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:55.923569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae5f6b-c566-418c-8bfd-221108e4e857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:55.924192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae5f6b-c566-418c-8bfd-221108e4e857 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:55.924487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae5f6b-c566-418c-8bfd-221108e4e857 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:55.926331 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae5f6b-c566-418c-8bfd-221108e4e857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:55.935169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae5f6b-c566-418c-8bfd-221108e4e857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:55.936656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae5f6b-c566-418c-8bfd-221108e4e857 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:55.946074 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1187/2347] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:56.168674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46d680-b67a-4560-9290-1b2a6f4e75b9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:56.179317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46d680-b67a-4560-9290-1b2a6f4e75b9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:56.183451 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:56.185232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:56.185232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:56.185232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:56.263684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:56.265197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:56.288070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1161/2348] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:41:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:41:56.400618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540085c-bf38-4314-bbce-75740e8d1a4a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:56.401834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540085c-bf38-4314-bbce-75740e8d1a4a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:56.402390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540085c-bf38-4314-bbce-75740e8d1a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:56.442709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9540085c-bf38-4314-bbce-75740e8d1a4a None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:41:56.445911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9540085c-bf38-4314-bbce-75740e8d1a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:56.537701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1188/2349] 10.222.0.128 () {58 vars in 972 bytes} [Tue Jul 26 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:41:56.658609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:41:56.660191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:41:56.745646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f001a6-9119-48e4-b4b8-ac323c8e5098 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:41:56.751183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21602dfd-0a53-4844-9ec8-7b1df275db1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:41:56.762865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21602dfd-0a53-4844-9ec8-7b1df275db1d service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:41:56.762865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21602dfd-0a53-4844-9ec8-7b1df275db1d service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:41:56.762865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21602dfd-0a53-4844-9ec8-7b1df275db1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:41:56.762865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21602dfd-0a53-4844-9ec8-7b1df275db1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:41:56.763941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21602dfd-0a53-4844-9ec8-7b1df275db1d service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:41:56.772768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1162/2350] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:41:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:42:01.253653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540085c-bf38-4314-bbce-75740e8d1a4a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:01.277752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:01.332530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540085c-bf38-4314-bbce-75740e8d1a4a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:01.339190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:01.340321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:01.340321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:01.340671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] PATH_INFO: `/v3/users/765fc2bd1b3f4112a640a37389a296a3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:01.342188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=765fc2bd1b3f4112a640a37389a296a3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:01.344581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:01.492730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765fc2bd1b3f4112a640a37389a296a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:01.493421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765fc2bd1b3f4112a640a37389a296a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:01.501020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765fc2bd1b3f4112a640a37389a296a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:01.522857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765fc2bd1b3f4112a640a37389a296a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:01.611862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765fc2bd1b3f4112a640a37389a296a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:01.623870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1189/2351] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:01 2022] DELETE /identity/v3/users/765fc2bd1b3f4112a640a37389a296a3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:01.625797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:01.626270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:01.626380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:01.651678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:01.653050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:01.724000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1163/2352] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:01.742101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:01.742476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:01.809727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d77231-0ca3-4943-b1f4-de4cdc55abf5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:01.813888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:01.814584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:01.814803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:01.815079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:01.829908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:01.831516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:01.845045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1190/2353] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:02.211134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:02.211134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:02.224754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:02.294504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e7200-9301-4d5f-9900-941f9a27ff9c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:02.299660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:02.300288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:02.300611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:02.300916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:02.303232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:02.316933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:02.318900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:02.330145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1164/2354] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:02.341495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:02.343283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:02.359857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d88ca-4815-489a-8e12-5888e604adc0 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:02.382652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:02.382652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:02.382652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:02.382652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] PATH_INFO: `/v3/projects/3096c2160750462cafacb887a1e0b6ac` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:02.383337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3096c2160750462cafacb887a1e0b6ac)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:02.383337 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:02.753664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:02.764373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3096c2160750462cafacb887a1e0b6ac', 'request_id': 'req-d4be63e8-b7b0-4136-91ce-db822851edf6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:02.771319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3096c2160750462cafacb887a1e0b6ac', 'request_id': 'req-d4be63e8-b7b0-4136-91ce-db822851edf6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:02.777938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3096c2160750462cafacb887a1e0b6ac', 'request_id': 'req-d4be63e8-b7b0-4136-91ce-db822851edf6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:02.898737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] The token cache is being invalidate because project 3096c2160750462cafacb887a1e0b6ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:02.899327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:02.900384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:02.902382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1191/2355] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:02 2022] DELETE /identity/v3/projects/3096c2160750462cafacb887a1e0b6ac => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:02.906179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:02.919154 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ba03-bd98-4f07-adce-67f0987a1658 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:02.990636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:02.990636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:02.990636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:02.990636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] PATH_INFO: `/v3/users/b51c3bb7385748bdb0468ba0c8833aa7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:02.992367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51c3bb7385748bdb0468ba0c8833aa7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:02.994620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:03.203247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51c3bb7385748bdb0468ba0c8833aa7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:03.215882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51c3bb7385748bdb0468ba0c8833aa7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:03.217849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51c3bb7385748bdb0468ba0c8833aa7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:03.268658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51c3bb7385748bdb0468ba0c8833aa7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:03.269609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d245d7c-a15b-4ece-a90f-07566af31814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51c3bb7385748bdb0468ba0c8833aa7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:03.291036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1165/2356] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:02 2022] DELETE /identity/v3/users/b51c3bb7385748bdb0468ba0c8833aa7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:04.283377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:04.285282 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:04.301473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be63e8-b7b0-4136-91ce-db822851edf6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:04.392866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:04.393600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:04.394027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:04.394325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] PATH_INFO: `/v3/projects/fac474c79f0f4bc68da46e48455c938b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:04.395543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fac474c79f0f4bc68da46e48455c938b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:04.397566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:04.520152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fac474c79f0f4bc68da46e48455c938b', 'request_id': 'req-2e99183c-4713-4d20-9e45-0a129966e0c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:04.527907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fac474c79f0f4bc68da46e48455c938b', 'request_id': 'req-2e99183c-4713-4d20-9e45-0a129966e0c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:04.544508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fac474c79f0f4bc68da46e48455c938b', 'request_id': 'req-2e99183c-4713-4d20-9e45-0a129966e0c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:04.613647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] The token cache is being invalidate because project fac474c79f0f4bc68da46e48455c938b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:04.613647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:04.613647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2e99183c-4713-4d20-9e45-0a129966e0c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:04.618719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1192/2357] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:04 2022] DELETE /identity/v3/projects/fac474c79f0f4bc68da46e48455c938b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:04.620367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f398ac-004a-433d-bd2a-708846dd0a40 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:04.620919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f398ac-004a-433d-bd2a-708846dd0a40 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:04.621097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f398ac-004a-433d-bd2a-708846dd0a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:04.649134 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-24f398ac-004a-433d-bd2a-708846dd0a40 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:04.651207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-24f398ac-004a-433d-bd2a-708846dd0a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:04.667133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:04.723682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1166/2358] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:04.732594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d84859-8f2d-447f-8865-1057457dd5d4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:04.732893 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d84859-8f2d-447f-8865-1057457dd5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:04.733156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d84859-8f2d-447f-8865-1057457dd5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:04.753855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-01d84859-8f2d-447f-8865-1057457dd5d4 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:04.755108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-01d84859-8f2d-447f-8865-1057457dd5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:04.764795 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1193/2359] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:04.769088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f398ac-004a-433d-bd2a-708846dd0a40 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:04.782082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f398ac-004a-433d-bd2a-708846dd0a40 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:04.791792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:04.791792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:04.791792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:04.791792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:04.792143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:42:04.792143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:04.793056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:04.823940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d84859-8f2d-447f-8865-1057457dd5d4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:04.831182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1167/2360] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:42:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:42:04.836643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d84859-8f2d-447f-8865-1057457dd5d4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:04.842582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:04.843825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:04.844422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:04.844965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:04.846481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:04.849108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:04.945732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1194/2361] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:04 2022] POST /identity/v3/projects => generated 382 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:04.953128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:04.963526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45f244-4e3c-4918-9681-573f8a8c123f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:05.025091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:05.025736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.026031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.026362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.027099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:05.029111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:05.152769 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1168/2362] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:04 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:05.154127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:05.173145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782adfe-cc3d-4ed3-bea0-049612e16b0c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:05.180085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:05.181051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.181385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.181699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.182352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:05.184009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:05.198273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1195/2363] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:05 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:05.199601 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:05.213886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b59561-3b67-4d08-a26e-52a269e8a9af admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:05.219935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271c086-1a26-46f8-83de-0dcc5a6a963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:05.220662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c086-1a26-46f8-83de-0dcc5a6a963f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.220959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c086-1a26-46f8-83de-0dcc5a6a963f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.221242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c086-1a26-46f8-83de-0dcc5a6a963f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.221970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271c086-1a26-46f8-83de-0dcc5a6a963f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:05.223976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271c086-1a26-46f8-83de-0dcc5a6a963f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:05.237733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1169/2364] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:05 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:05.242606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:05.266056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e1c67-29ea-4c56-83f7-c563c6971aee admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:05.272077 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:05.272973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.273334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.273627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] PATH_INFO: `/v3/projects/1ada76cb4c214395a5d7a618859f35e3/users/55caf4b1e9c94303ad54348785abf0ae/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.293393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:05.297758 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ada76cb4c214395a5d7a618859f35e3, user_id=55caf4b1e9c94303ad54348785abf0ae, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:05.300643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:05.403734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1196/2365] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:05 2022] PUT /identity/v3/projects/1ada76cb4c214395a5d7a618859f35e3/users/55caf4b1e9c94303ad54348785abf0ae/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:05.419434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1bf2b-4253-4795-963b-e04f44d2b45b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.419792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1bf2b-4253-4795-963b-e04f44d2b45b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.419999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1bf2b-4253-4795-963b-e04f44d2b45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.454417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-97e1bf2b-4253-4795-963b-e04f44d2b45b None None] MFA Rules not processed for user `55caf4b1e9c94303ad54348785abf0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:05.455854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97e1bf2b-4253-4795-963b-e04f44d2b45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:05.538122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1170/2366] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:05.543441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:05.613824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ab3d-5f17-4f92-a7ef-a901b2671ffb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:05.621057 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:05.621888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.622343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.622715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.623605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:05.627111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:05.717311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1197/2367] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:05 2022] POST /identity/v3/projects => generated 384 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:42:05.722229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1bf2b-4253-4795-963b-e04f44d2b45b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:05.741326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1bf2b-4253-4795-963b-e04f44d2b45b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:05.756660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:05.813513 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:05.814138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.814421 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.814742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.816514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:05.818401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:05.925439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1171/2368] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:05 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:05.926800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:05.943820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903059-0074-4ad7-b51a-f83009f8053a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:05.958196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:05.958196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:05.958196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:05.958196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:05.958857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:05.963767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:05.991638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:05.992500 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1198/2369] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:05 2022] GET /identity/v3/roles => generated 1427 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:06.023667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3876-9e28-4cd6-bdc7-b27284d61d3d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:06.030095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:06.030964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:06.031360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:06.031748 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] PATH_INFO: `/v3/projects/0d718ce0cf2c448a8ccc42f8f312743a/users/ec5fcca62f974745a3dcb0e1e06ea55b/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:06.077467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d718ce0cf2c448a8ccc42f8f312743a, user_id=ec5fcca62f974745a3dcb0e1e06ea55b, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:06.083688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:06.199588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1172/2370] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:05 2022] PUT /identity/v3/projects/0d718ce0cf2c448a8ccc42f8f312743a/users/ec5fcca62f974745a3dcb0e1e06ea55b/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:06.200791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:06.217229 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0c41d-3608-4cc9-8c74-767e3a3426e0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:06.295454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787a568-1957-4210-be6a-7fcdab619c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:06.296085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787a568-1957-4210-be6a-7fcdab619c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:06.296388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787a568-1957-4210-be6a-7fcdab619c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:06.296728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787a568-1957-4210-be6a-7fcdab619c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:06.297431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787a568-1957-4210-be6a-7fcdab619c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:06.299049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787a568-1957-4210-be6a-7fcdab619c0e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:06.312329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1199/2371] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:06 2022] GET /identity/v3/roles => generated 1427 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:42:06.317126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:06.335944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788c25-450a-4ee1-9830-df7d5fb4690f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:06.367002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:06.369746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:06.370242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:06.370720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] PATH_INFO: `/v3/projects/0d718ce0cf2c448a8ccc42f8f312743a/users/ec5fcca62f974745a3dcb0e1e06ea55b/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:06.375558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d718ce0cf2c448a8ccc42f8f312743a, user_id=ec5fcca62f974745a3dcb0e1e06ea55b, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:06.381439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:06.423213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1173/2372] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:06 2022] PUT /identity/v3/projects/0d718ce0cf2c448a8ccc42f8f312743a/users/ec5fcca62f974745a3dcb0e1e06ea55b/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:06.482517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:06.482817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:06.483100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:06.521592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] MFA Rules not processed for user `ec5fcca62f974745a3dcb0e1e06ea55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:06.523356 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:06.602100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:06.634174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1200/2373] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:06.649262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:06.651125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:06.721447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:06.723697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:06.739608 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc692284-bb07-44e4-a1d9-b80c6b2fd441 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:06.747865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:06.747865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:06.747865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:06.748175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:06.759161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:06.760817 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:06.772430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1174/2374] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:06.810141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39a413-9461-4042-9acb-be30ee3b8d12 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:06.814858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:06.815619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:06.815966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:06.816315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] PATH_INFO: `/v3/users/1f6f23b2e4134aa78862287f57323731` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:06.817810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6f23b2e4134aa78862287f57323731)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:06.821176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:06.928387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:06.937646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07228f6-e190-4b0b-b65e-d106447b6de7 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:06.941306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:06.942034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:06.942320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:06.942598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:07.004976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:07.006442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:07.021112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1175/2375] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:42:07.027966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6f23b2e4134aa78862287f57323731'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.027966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6f23b2e4134aa78862287f57323731'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.039335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6f23b2e4134aa78862287f57323731'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.067121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6f23b2e4134aa78862287f57323731'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.125439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6f23b2e4134aa78862287f57323731'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.134394 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1201/2376] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:06 2022] DELETE /identity/v3/users/1f6f23b2e4134aa78862287f57323731 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:07.432382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:07.446163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e7d3-baab-4ccb-b286-e1fb4dfe503b service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:07.451192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:07.513287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:07.514774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:07.543806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:07.544599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:07.545081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:07.545434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] PATH_INFO: `/v3/projects/7c892ef56ae9488089386d1a733ae3c0` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:07.546897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c892ef56ae9488089386d1a733ae3c0)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:07.550346 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:07.614187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3b527-f679-440f-8b2f-fd5464621fa4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:07.619755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:07.620601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:07.620890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:07.621172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] PATH_INFO: `/v3/users/55caf4b1e9c94303ad54348785abf0ae` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:07.622477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] RBAC: Authorizing `identity:delete_user(user_id=55caf4b1e9c94303ad54348785abf0ae)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:07.624622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:07.630125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:07.674430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c892ef56ae9488089386d1a733ae3c0', 'request_id': 'req-6a8d1e15-b814-4442-915e-cd85904e0027'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.759871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c892ef56ae9488089386d1a733ae3c0', 'request_id': 'req-6a8d1e15-b814-4442-915e-cd85904e0027'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.768157 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c892ef56ae9488089386d1a733ae3c0', 'request_id': 'req-6a8d1e15-b814-4442-915e-cd85904e0027'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.781852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] The token cache is being invalidate because project 7c892ef56ae9488089386d1a733ae3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:07.782515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.783717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.786038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1176/2377] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:07 2022] DELETE /identity/v3/projects/7c892ef56ae9488089386d1a733ae3c0 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:07.791978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:07.793747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:07.825875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55caf4b1e9c94303ad54348785abf0ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.826755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55caf4b1e9c94303ad54348785abf0ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.840173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55caf4b1e9c94303ad54348785abf0ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.861154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55caf4b1e9c94303ad54348785abf0ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.876845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d1e15-b814-4442-915e-cd85904e0027 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:07.883172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:07.886066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:07.886066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:07.886066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] PATH_INFO: `/v3/users/5b2c1d60c19148a6ad2f893b0c00cdac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:07.886614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2c1d60c19148a6ad2f893b0c00cdac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:07.889003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:07.925221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a48953d-c92e-4f26-b4f2-d9d2730aceab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55caf4b1e9c94303ad54348785abf0ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:07.933458 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1202/2378] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:07 2022] DELETE /identity/v3/users/55caf4b1e9c94303ad54348785abf0ae => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:07.938341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:07.938635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:07.938917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:07.962856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:07.964244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:08.056618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1203/2379] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:08.065604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:08.067524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:08.093751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:08.110338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2c1d60c19148a6ad2f893b0c00cdac'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.120717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2c1d60c19148a6ad2f893b0c00cdac'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.121612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2c1d60c19148a6ad2f893b0c00cdac'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.140767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80df50-62cd-4531-9b77-028a3b9f5b7a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:08.145121 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:08.145986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:08.146366 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:08.146571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:08.149241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:08.162270 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:08.165256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:08.176899 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1204/2380] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:42:08.199442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2c1d60c19148a6ad2f893b0c00cdac'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.200254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2c1d60c19148a6ad2f893b0c00cdac'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.229917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1177/2381] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:07 2022] DELETE /identity/v3/users/5b2c1d60c19148a6ad2f893b0c00cdac => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:08.586323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:08.588209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:08.613264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0e7ac-3d76-4698-af7f-b61c569ed33a service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:08.641709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:08.676782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:08.677766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:08.678050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:08.678328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] PATH_INFO: `/v3/projects/94a7db1ba4cf4b0d9a824c148c641109` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:08.680205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a7db1ba4cf4b0d9a824c148c641109)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:08.682226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:08.700388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:08.703069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:08.717973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfe74d-fd33-4a58-b9cb-f6e2f161b9de admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:08.725273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:08.726648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:08.726944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:08.727403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] PATH_INFO: `/v3/projects/1ada76cb4c214395a5d7a618859f35e3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:08.729050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ada76cb4c214395a5d7a618859f35e3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:08.730954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:08.817855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a7db1ba4cf4b0d9a824c148c641109', 'request_id': 'req-c20995cf-33eb-4def-8de7-29f9d8be3683'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.824952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a7db1ba4cf4b0d9a824c148c641109', 'request_id': 'req-c20995cf-33eb-4def-8de7-29f9d8be3683'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.830531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a7db1ba4cf4b0d9a824c148c641109', 'request_id': 'req-c20995cf-33eb-4def-8de7-29f9d8be3683'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.863683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ada76cb4c214395a5d7a618859f35e3', 'request_id': 'req-ff9e1439-e3fc-4e70-9882-57e351c52c64'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.937327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] The token cache is being invalidate because project 94a7db1ba4cf4b0d9a824c148c641109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:08.937622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.938292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:08.939849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1205/2382] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:08 2022] DELETE /identity/v3/projects/94a7db1ba4cf4b0d9a824c148c641109 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:08.999351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ada76cb4c214395a5d7a618859f35e3', 'request_id': 'req-ff9e1439-e3fc-4e70-9882-57e351c52c64'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.006090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ada76cb4c214395a5d7a618859f35e3', 'request_id': 'req-ff9e1439-e3fc-4e70-9882-57e351c52c64'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.021061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] The token cache is being invalidate because project 1ada76cb4c214395a5d7a618859f35e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:09.021920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.022906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.024784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1178/2383] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:08 2022] DELETE /identity/v3/projects/1ada76cb4c214395a5d7a618859f35e3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:09.029850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:09.031324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:09.107902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20995cf-33eb-4def-8de7-29f9d8be3683 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:09.182753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:09.183421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:09.183702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:09.184004 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] PATH_INFO: `/v3/users/ec5fcca62f974745a3dcb0e1e06ea55b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:09.185509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5fcca62f974745a3dcb0e1e06ea55b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:09.187623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:09.192945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:09.345547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5fcca62f974745a3dcb0e1e06ea55b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.346275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5fcca62f974745a3dcb0e1e06ea55b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.353222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5fcca62f974745a3dcb0e1e06ea55b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.365819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5fcca62f974745a3dcb0e1e06ea55b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.456846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-347a9382-a336-413d-b84f-5f65b697bfbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5fcca62f974745a3dcb0e1e06ea55b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:09.463796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1206/2384] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:09 2022] DELETE /identity/v3/users/ec5fcca62f974745a3dcb0e1e06ea55b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:09.955776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:09.957380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:09.972974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1439-e3fc-4e70-9882-57e351c52c64 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:10.043238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:10.043947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.044285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.044600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] PATH_INFO: `/v3/projects/0d718ce0cf2c448a8ccc42f8f312743a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.045811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d718ce0cf2c448a8ccc42f8f312743a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:10.047727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:10.125567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:10.131554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d718ce0cf2c448a8ccc42f8f312743a', 'request_id': 'req-606eef91-2b6e-4d94-b2d3-c86b0d930604'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:10.176256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d718ce0cf2c448a8ccc42f8f312743a', 'request_id': 'req-606eef91-2b6e-4d94-b2d3-c86b0d930604'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:10.183230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d718ce0cf2c448a8ccc42f8f312743a', 'request_id': 'req-606eef91-2b6e-4d94-b2d3-c86b0d930604'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:10.195841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] The token cache is being invalidate because project 0d718ce0cf2c448a8ccc42f8f312743a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:10.196132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:10.196840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-606eef91-2b6e-4d94-b2d3-c86b0d930604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:10.198490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1179/2385] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:09 2022] DELETE /identity/v3/projects/0d718ce0cf2c448a8ccc42f8f312743a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:10.203438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78bdb5-06ff-49eb-8598-de1cd29f40f2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.203910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78bdb5-06ff-49eb-8598-de1cd29f40f2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.204011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78bdb5-06ff-49eb-8598-de1cd29f40f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.224163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fe78bdb5-06ff-49eb-8598-de1cd29f40f2 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:10.225388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe78bdb5-06ff-49eb-8598-de1cd29f40f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:10.291813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1207/2386] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:10.299664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce67d84-7eb8-4265-bd99-238eda570423 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.299955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce67d84-7eb8-4265-bd99-238eda570423 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.300264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce67d84-7eb8-4265-bd99-238eda570423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.320196 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8ce67d84-7eb8-4265-bd99-238eda570423 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:10.321458 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ce67d84-7eb8-4265-bd99-238eda570423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:10.329337 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1180/2387] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:10.333627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78bdb5-06ff-49eb-8598-de1cd29f40f2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:10.343711 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78bdb5-06ff-49eb-8598-de1cd29f40f2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:10.348243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:10.348871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.349154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.349476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.350203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:42:10.350506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:10.352880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:10.364246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1208/2388] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:42:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:42:10.368905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce67d84-7eb8-4265-bd99-238eda570423 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:10.378742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce67d84-7eb8-4265-bd99-238eda570423 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:10.384970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:10.385678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.385961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.388902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.388902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:10.388902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:10.469649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1181/2389] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:10 2022] POST /identity/v3/projects => generated 372 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:10.474084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:10.484961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ba4a-f78b-47dc-8f5b-c8949f15f123 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:10.546016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:10.546642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.546928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.547205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.547881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:10.549856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:10.628993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:10.639191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1209/2390] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:10 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:10.639594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:10.651671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdb2fc-8d22-428f-af47-4ecb5efd6fe5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:10.656625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:10.657281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.657567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.657975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.658686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:10.660398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:10.671395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1182/2391] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:10 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:10.675230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:10.687869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae265b-d1d6-4b90-9faf-145f505a23cf admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:10.692677 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcd886-1bbc-4b93-968d-42e155881b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:10.693445 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcd886-1bbc-4b93-968d-42e155881b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.693725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcd886-1bbc-4b93-968d-42e155881b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.693999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcd886-1bbc-4b93-968d-42e155881b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.695799 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcd886-1bbc-4b93-968d-42e155881b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:10.697448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcd886-1bbc-4b93-968d-42e155881b05 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:10.708992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1210/2392] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:10 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:10.714579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:10.724931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27571400-beda-45ff-83bd-bee9d9da262a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:10.730368 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:10.731120 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.731413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.731701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] PATH_INFO: `/v3/projects/c2045e25750c4a648c076629cb6dcf96/users/ea4425e36f6c4fa18f20dca383ebd44d/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.747153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2045e25750c4a648c076629cb6dcf96, user_id=ea4425e36f6c4fa18f20dca383ebd44d, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:10.749933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:10.829267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1183/2393] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:10 2022] PUT /identity/v3/projects/c2045e25750c4a648c076629cb6dcf96/users/ea4425e36f6c4fa18f20dca383ebd44d/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:10.836492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1196-734b-4c67-bcee-0f50a7ea180f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:10.836789 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1196-734b-4c67-bcee-0f50a7ea180f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:10.837071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1196-734b-4c67-bcee-0f50a7ea180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:10.856718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d5ce1196-734b-4c67-bcee-0f50a7ea180f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:10.857938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d5ce1196-734b-4c67-bcee-0f50a7ea180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:10.922523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1211/2394] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:10.934288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:10.936047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:11.000230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d641-dcfe-47af-a9a4-5cdc23865928 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:11.005252 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:11.005799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:11.006086 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:11.006360 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:11.016633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:11.018075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:11.028145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1184/2395] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:12.612871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e989b1-fcab-4af0-b2f9-757bd7ab3e3e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:12.613438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e989b1-fcab-4af0-b2f9-757bd7ab3e3e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:12.613558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e989b1-fcab-4af0-b2f9-757bd7ab3e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:12.646333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-42e989b1-fcab-4af0-b2f9-757bd7ab3e3e None None] MFA Rules not processed for user `ea4425e36f6c4fa18f20dca383ebd44d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:12.648891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-42e989b1-fcab-4af0-b2f9-757bd7ab3e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:12.721778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1212/2396] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:12.727750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:12.737743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e727a5-6c9f-42b6-be03-cc018778e8ea service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:12.743834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:12.744696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:12.744990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:12.745270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:12.746038 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:12.748040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:12.842455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1185/2397] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:12 2022] POST /identity/v3/projects => generated 374 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:12.847260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e989b1-fcab-4af0-b2f9-757bd7ab3e3e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:12.865924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e989b1-fcab-4af0-b2f9-757bd7ab3e3e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:12.927803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:12.928435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:12.929877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:12.931086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:12.932763 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:12.934210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:13.051811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1213/2398] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:12 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:13.057419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:13.068114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf7923-4386-44d3-aac6-98dcc71c2a00 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:13.073478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:13.074092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:13.074377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:13.074658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:13.075389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:13.077026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:13.088301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1186/2399] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:13 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:42:13.092740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:13.103896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa787b6-223b-40d4-b252-19ce55eab8f0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:13.109679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:13.110363 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:13.110653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:13.110936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] PATH_INFO: `/v3/projects/6941088071c14ec5b8f27e04923ed69e/users/5be626ff4cf244af814278b962451c26/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:13.114461 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:13.128201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6941088071c14ec5b8f27e04923ed69e, user_id=5be626ff4cf244af814278b962451c26, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:13.131106 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:13.211920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1214/2400] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:13 2022] PUT /identity/v3/projects/6941088071c14ec5b8f27e04923ed69e/users/5be626ff4cf244af814278b962451c26/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:13.216331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:13.227310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167631e-c924-4102-9a71-8b0df3f51a5f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:13.260289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:13.287693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593b251-0199-4868-8767-b37cf99829a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:13.289189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593b251-0199-4868-8767-b37cf99829a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:13.289483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593b251-0199-4868-8767-b37cf99829a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:13.289765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593b251-0199-4868-8767-b37cf99829a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:13.290479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593b251-0199-4868-8767-b37cf99829a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:13.292241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593b251-0199-4868-8767-b37cf99829a2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:13.303997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1187/2401] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:13 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:13.309787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:13.329600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c20df-4aec-4a0c-9441-a5c948cd81f3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:13.334335 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:13.335553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:13.335829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:13.336111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] PATH_INFO: `/v3/projects/6941088071c14ec5b8f27e04923ed69e/users/5be626ff4cf244af814278b962451c26/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:13.338201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6941088071c14ec5b8f27e04923ed69e, user_id=5be626ff4cf244af814278b962451c26, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:13.340896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:13.358665 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1215/2402] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:13 2022] PUT /identity/v3/projects/6941088071c14ec5b8f27e04923ed69e/users/5be626ff4cf244af814278b962451c26/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:14.663371 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:14.664189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:14.664189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:14.703546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] MFA Rules not processed for user `5be626ff4cf244af814278b962451c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:14.704819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:14.814945 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1188/2403] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:14.823218 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:14.826136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:14.903828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f4f17-53d2-414e-b663-9556265a15ba admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:14.908207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:14.908886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:14.909172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:14.909453 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:14.969839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:14.971339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:14.981992 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1216/2404] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:16.767573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:16.768654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:16.782246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f410ec3-202f-45c9-a12b-e4eaa9a8ac4c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:16.786267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b36a5-8646-420b-b42c-0fc3d6631ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:16.786902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b36a5-8646-420b-b42c-0fc3d6631ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:16.787189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b36a5-8646-420b-b42c-0fc3d6631ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:16.787471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b36a5-8646-420b-b42c-0fc3d6631ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:16.801168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b36a5-8646-420b-b42c-0fc3d6631ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:16.802888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b36a5-8646-420b-b42c-0fc3d6631ba0 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:16.814468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1189/2405] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:24.977561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=59,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:26.809756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=53 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:32.982372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:33.051602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b30e86-de3d-4dc9-b86d-0772f6314da2 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:33.056513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:33.057850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:33.058415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:33.058415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] PATH_INFO: `/v3/users/ea4425e36f6c4fa18f20dca383ebd44d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:33.060070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4425e36f6c4fa18f20dca383ebd44d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:33.061826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:33.235049 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4425e36f6c4fa18f20dca383ebd44d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:33.235642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4425e36f6c4fa18f20dca383ebd44d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:33.242589 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4425e36f6c4fa18f20dca383ebd44d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:33.255280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4425e36f6c4fa18f20dca383ebd44d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:33.369355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4425e36f6c4fa18f20dca383ebd44d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:33.376402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1217/2406] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:32 2022] DELETE /identity/v3/users/ea4425e36f6c4fa18f20dca383ebd44d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:33.663057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:33.663057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:33.663057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:33.663057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:33.663057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:33.715074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1190/2407] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:42:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:33.720738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:33.723225 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:33.735194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:33.736938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:33.808021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0de8-c082-45fb-9c7b-6cbd07c86c75 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:33.812924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:33.813649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:33.813935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:33.814216 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] PATH_INFO: `/v3/projects/c2045e25750c4a648c076629cb6dcf96` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:33.815447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2045e25750c4a648c076629cb6dcf96)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:33.817588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:33.817738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeadd0-617d-4f84-a80b-c485270e826f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:33.823345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:33.832389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:33.832389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:33.832389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:33.834935 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:33.837977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:33.843821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:33.849505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1191/2408] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:42:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:42:33.955210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2045e25750c4a648c076629cb6dcf96', 'request_id': 'req-6be08d3b-9921-42da-a6f6-774729a024fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:33.962003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2045e25750c4a648c076629cb6dcf96', 'request_id': 'req-6be08d3b-9921-42da-a6f6-774729a024fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:33.968804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2045e25750c4a648c076629cb6dcf96', 'request_id': 'req-6be08d3b-9921-42da-a6f6-774729a024fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.149466 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] The token cache is being invalidate because project c2045e25750c4a648c076629cb6dcf96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:34.150478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.151150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.157056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:34.158231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1218/2409] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:33 2022] DELETE /identity/v3/projects/c2045e25750c4a648c076629cb6dcf96 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:34.171663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18116-913f-4ef6-a013-8aa1be5bcbdc service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:34.253396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:34.254059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:34.254348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:34.254626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] PATH_INFO: `/v3/users/5be626ff4cf244af814278b962451c26` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:34.255941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be626ff4cf244af814278b962451c26)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:34.257922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:34.494505 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:34.508145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be626ff4cf244af814278b962451c26'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.533467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be626ff4cf244af814278b962451c26'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.534589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be626ff4cf244af814278b962451c26'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.614853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be626ff4cf244af814278b962451c26'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.615761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-677d451e-49d2-41c9-8eb5-ac21038952b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be626ff4cf244af814278b962451c26'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:34.651699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1192/2410] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:34 2022] DELETE /identity/v3/users/5be626ff4cf244af814278b962451c26 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:34.952946 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:34.954175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:34.973989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be08d3b-9921-42da-a6f6-774729a024fd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:35.073517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:35.074386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.074668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.074956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] PATH_INFO: `/v3/projects/6941088071c14ec5b8f27e04923ed69e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.076723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6941088071c14ec5b8f27e04923ed69e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:35.078683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:35.226455 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6941088071c14ec5b8f27e04923ed69e', 'request_id': 'req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:35.235286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6941088071c14ec5b8f27e04923ed69e', 'request_id': 'req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:35.249687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6941088071c14ec5b8f27e04923ed69e', 'request_id': 'req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:35.346869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] The token cache is being invalidate because project 6941088071c14ec5b8f27e04923ed69e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:35.347169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:35.347989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52148b0c-5ad3-4ff6-8de7-0b4d0ca8c262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:35.350005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1219/2411] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:34 2022] DELETE /identity/v3/projects/6941088071c14ec5b8f27e04923ed69e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:35.356257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fdc4e-de02-464a-a326-4d262709c621 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.357595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fdc4e-de02-464a-a326-4d262709c621 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.357788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fdc4e-de02-464a-a326-4d262709c621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.382762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d93fdc4e-de02-464a-a326-4d262709c621 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:35.384015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d93fdc4e-de02-464a-a326-4d262709c621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:35.470625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1193/2412] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:35.480191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219de6f-29b7-42d0-8b74-25c63ef06792 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.480501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219de6f-29b7-42d0-8b74-25c63ef06792 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.480819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219de6f-29b7-42d0-8b74-25c63ef06792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.505605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3219de6f-29b7-42d0-8b74-25c63ef06792 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:35.506757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3219de6f-29b7-42d0-8b74-25c63ef06792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:35.523286 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1220/2413] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:35.524847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fdc4e-de02-464a-a326-4d262709c621 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:35.538998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fdc4e-de02-464a-a326-4d262709c621 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:35.545359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:35.546063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.546468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.546838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.547620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:42:35.548324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:35.551572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:35.573121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1194/2414] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:42:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:42:35.574294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219de6f-29b7-42d0-8b74-25c63ef06792 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:35.584731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219de6f-29b7-42d0-8b74-25c63ef06792 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:35.592704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:35.593355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.593641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.593924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.594610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:35.596836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:35.701893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:35.711323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1221/2415] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:35 2022] POST /identity/v3/projects => generated 384 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:42:35.717790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bb511-accd-48c4-84e4-3784a25f8631 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:35.785517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:35.786534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.786834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.787262 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.787962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:35.789976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:35.862873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1195/2416] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:35 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:35.867288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:35.882407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f19c7-33d3-4a8c-91a4-5f20962976fe admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:35.888982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:35.890211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.890211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.890420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.890947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:35.892585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:35.898477 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:35.907496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1222/2417] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:35 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:35.913450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:35.922847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f87c-1b4f-4d25-9584-4ba76894cb17 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:35.928903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9919386-69bd-471e-b3db-ec016de98ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:35.934365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919386-69bd-471e-b3db-ec016de98ced admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.934365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919386-69bd-471e-b3db-ec016de98ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.934365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919386-69bd-471e-b3db-ec016de98ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.934365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9919386-69bd-471e-b3db-ec016de98ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:35.934365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9919386-69bd-471e-b3db-ec016de98ced admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:35.946048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1196/2418] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:35 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:42:35.951365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:35.962566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a8f8-a395-4181-90b6-cc0644a5f6e3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:35.968055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:35.968896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:35.974484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:35.974484 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] PATH_INFO: `/v3/projects/547f8284fbe14b4aa03141adb0677708/users/911602f085a0445f8b78b9fd3a16b467/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:35.986573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] RBAC: Authorizing `identity:create_grant(project_id=547f8284fbe14b4aa03141adb0677708, user_id=911602f085a0445f8b78b9fd3a16b467, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:35.989895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:36.140995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1223/2419] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:35 2022] PUT /identity/v3/projects/547f8284fbe14b4aa03141adb0677708/users/911602f085a0445f8b78b9fd3a16b467/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:36.147677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f139da-827b-47dd-9371-37f1bc66eb56 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:36.148174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f139da-827b-47dd-9371-37f1bc66eb56 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:36.148333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f139da-827b-47dd-9371-37f1bc66eb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:36.186732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-94f139da-827b-47dd-9371-37f1bc66eb56 None None] MFA Rules not processed for user `911602f085a0445f8b78b9fd3a16b467`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:36.187966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-94f139da-827b-47dd-9371-37f1bc66eb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:36.199236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:36.306463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1197/2420] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:36.306686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:36.409092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036c4ee-8955-4a17-81f8-b091e32b481f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:36.415064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:36.425907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:36.425907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:36.425907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:36.425907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:36.425907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:36.525578 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1224/2421] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:36 2022] POST /identity/v3/projects => generated 384 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:42:36.527016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f139da-827b-47dd-9371-37f1bc66eb56 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:36.538245 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f139da-827b-47dd-9371-37f1bc66eb56 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:36.607305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:36.607900 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:36.608188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:36.608489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:36.609179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:36.611271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:36.734712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:36.735925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1198/2422] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:36 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:42:36.747592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb1473-7b6b-4f4b-961e-8840997505d7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:36.753809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:36.754452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:36.754741 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:36.755062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:36.756341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:36.758847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:36.780682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1225/2423] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:36 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:42:36.784610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:36.797284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfc3c-0843-4e63-b121-014c6d5757ff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:36.806909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc93c24-e18e-4396-a6ed-599519a20c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:36.811222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc93c24-e18e-4396-a6ed-599519a20c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:36.811222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc93c24-e18e-4396-a6ed-599519a20c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:36.811222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc93c24-e18e-4396-a6ed-599519a20c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:36.811222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc93c24-e18e-4396-a6ed-599519a20c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:36.812803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc93c24-e18e-4396-a6ed-599519a20c23 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:36.826359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1199/2424] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:36 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:42:36.831021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:36.843064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2827be-1c52-488b-b4ba-5b063fd99f41 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:36.847751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:36.848444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:36.848756 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:36.849039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] PATH_INFO: `/v3/projects/860e687f576846cfbfca5dac9a99b067/users/d566a1c7f0284fb1b390e05f5d2507f9/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:36.872816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=860e687f576846cfbfca5dac9a99b067, user_id=d566a1c7f0284fb1b390e05f5d2507f9, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:36.872816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:36.970508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1226/2425] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:36 2022] PUT /identity/v3/projects/860e687f576846cfbfca5dac9a99b067/users/d566a1c7f0284fb1b390e05f5d2507f9/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:36.981871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbdf7b-5416-49eb-8676-3747e22d701a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:36.982165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbdf7b-5416-49eb-8676-3747e22d701a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:36.982454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbdf7b-5416-49eb-8676-3747e22d701a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:37.015183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b7dbdf7b-5416-49eb-8676-3747e22d701a None None] MFA Rules not processed for user `d566a1c7f0284fb1b390e05f5d2507f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:37.016483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b7dbdf7b-5416-49eb-8676-3747e22d701a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:37.094820 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1200/2426] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:37.099361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:37.171402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98811b28-3aa0-4fa4-b2ae-acbf7969d00a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:37.176304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:37.176955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:37.177235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:37.177511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:37.178192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:37.180123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:37.502086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1227/2427] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:37 2022] POST /identity/v3/projects => generated 386 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:37.502653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbdf7b-5416-49eb-8676-3747e22d701a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:37.510495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbdf7b-5416-49eb-8676-3747e22d701a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:37.524868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:37.571633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:37.573327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:37.573327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:37.573556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:37.574248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:37.576201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:37.701272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1201/2428] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:37 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:42:37.705557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:37.717015 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40aa5c-2e55-4fc5-a42d-15573bdb2f40 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:37.721650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:37.722261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:37.722538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:37.722813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:37.723509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:37.725147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:37.731364 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:37.743092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1228/2429] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:37 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:42:37.743367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:37.753914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fa661-9288-4275-bf7e-2d1a9e37fde9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:37.759739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:37.761673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:37.770162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:37.772051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] PATH_INFO: `/v3/projects/570de1faf53446ba9a6f27cc937e6eca/users/e27d872fd21e4517958e1550abb706f8/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:37.810014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=570de1faf53446ba9a6f27cc937e6eca, user_id=e27d872fd21e4517958e1550abb706f8, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:37.812754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:37.930482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1202/2430] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:37 2022] PUT /identity/v3/projects/570de1faf53446ba9a6f27cc937e6eca/users/e27d872fd21e4517958e1550abb706f8/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:37.936173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:37.947079 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a4f0f-5bbc-4293-98b3-312682a243e2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:38.043173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724057be-3bc3-4434-ab88-fe5c708b238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:38.047884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724057be-3bc3-4434-ab88-fe5c708b238d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:38.047884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724057be-3bc3-4434-ab88-fe5c708b238d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:38.047884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724057be-3bc3-4434-ab88-fe5c708b238d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:38.047884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724057be-3bc3-4434-ab88-fe5c708b238d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:38.051947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724057be-3bc3-4434-ab88-fe5c708b238d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:38.065009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1229/2431] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:37 2022] GET /identity/v3/roles => generated 1427 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:38.068870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:38.082756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa1730-1a2a-4040-8856-a9c86b404850 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:38.087977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:38.089329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:38.089790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:38.090061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] PATH_INFO: `/v3/projects/570de1faf53446ba9a6f27cc937e6eca/users/e27d872fd21e4517958e1550abb706f8/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:38.092294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=570de1faf53446ba9a6f27cc937e6eca, user_id=e27d872fd21e4517958e1550abb706f8, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:38.095063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:38.117074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1203/2432] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:38 2022] PUT /identity/v3/projects/570de1faf53446ba9a6f27cc937e6eca/users/e27d872fd21e4517958e1550abb706f8/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:38.124150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7c782-dc00-4958-80fd-4a7360a3045f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:38.124150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7c782-dc00-4958-80fd-4a7360a3045f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:38.124150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7c782-dc00-4958-80fd-4a7360a3045f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:38.160564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3fa7c782-dc00-4958-80fd-4a7360a3045f None None] MFA Rules not processed for user `e27d872fd21e4517958e1550abb706f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:38.161910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3fa7c782-dc00-4958-80fd-4a7360a3045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:38.288635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1230/2433] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:38.299816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:38.304697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:38.392416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b39f73-0789-4b50-9701-e67d3bfe3e9f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:38.398320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:38.399266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:38.399754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:38.400255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:38.493187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:38.495558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:38.506767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1204/2434] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:40.853065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7c782-dc00-4958-80fd-4a7360a3045f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:40.863474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7c782-dc00-4958-80fd-4a7360a3045f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:40.867103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:40.867725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:40.868011 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:40.868290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:40.929019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:40.930535 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:40.940124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1231/2435] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:42.539251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:42.551553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e36717-d197-4ce6-bddf-bd84d65d1f48 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:42.560406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ff44f-40d6-43f9-98d7-82fc6e7e43bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:42.561136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ff44f-40d6-43f9-98d7-82fc6e7e43bc service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:42.561476 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ff44f-40d6-43f9-98d7-82fc6e7e43bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:42.561772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ff44f-40d6-43f9-98d7-82fc6e7e43bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:42.575279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ff44f-40d6-43f9-98d7-82fc6e7e43bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:42.576824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ff44f-40d6-43f9-98d7-82fc6e7e43bc service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:42.587734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1205/2436] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:47.448828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:47.471145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:47.515310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccc25a-7253-45f7-963e-5d7ccee6cb97 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:47.520001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:47.520678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:47.520968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:47.521247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] PATH_INFO: `/v3/users/911602f085a0445f8b78b9fd3a16b467` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:47.522411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=911602f085a0445f8b78b9fd3a16b467)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:47.524579 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:47.765150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '911602f085a0445f8b78b9fd3a16b467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:47.765812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '911602f085a0445f8b78b9fd3a16b467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:47.773813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '911602f085a0445f8b78b9fd3a16b467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:47.787725 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '911602f085a0445f8b78b9fd3a16b467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:47.876182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '911602f085a0445f8b78b9fd3a16b467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:47.891269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1232/2437] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:47 2022] DELETE /identity/v3/users/911602f085a0445f8b78b9fd3a16b467 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:47.898954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:47.899480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:47.899596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:47.921350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:47.922582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:47.928276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:47.986650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1206/2438] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:47.996509 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:47.997923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:48.057823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1fce2-5b00-4821-be92-f513c7a0cae8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:48.062083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:48.062698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:48.062978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:48.063251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:48.072247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:48.075241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:48.084970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1233/2439] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:48.473941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:48.475718 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:48.492751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828a4ef-19df-44d2-adf2-16d130877e59 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:48.499857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:48.500733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:48.501061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:48.501394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] PATH_INFO: `/v3/projects/547f8284fbe14b4aa03141adb0677708` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:48.502706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547f8284fbe14b4aa03141adb0677708)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:48.505240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:48.651478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547f8284fbe14b4aa03141adb0677708', 'request_id': 'req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:48.711189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547f8284fbe14b4aa03141adb0677708', 'request_id': 'req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:48.722213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547f8284fbe14b4aa03141adb0677708', 'request_id': 'req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:48.736012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] The token cache is being invalidate because project 547f8284fbe14b4aa03141adb0677708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:48.736322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:48.737195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:48.738986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1207/2440] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:48 2022] DELETE /identity/v3/projects/547f8284fbe14b4aa03141adb0677708 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:48.742739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:48.757220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d678c4-85b8-4c30-b412-455964cd758e service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:48.820212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:48.840512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:48.841264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:48.841551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:48.841951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] PATH_INFO: `/v3/users/d566a1c7f0284fb1b390e05f5d2507f9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:48.843390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d566a1c7f0284fb1b390e05f5d2507f9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:48.845644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:49.030438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd566a1c7f0284fb1b390e05f5d2507f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:49.031996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd566a1c7f0284fb1b390e05f5d2507f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:49.048890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd566a1c7f0284fb1b390e05f5d2507f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:49.074000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd566a1c7f0284fb1b390e05f5d2507f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:49.139695 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd566a1c7f0284fb1b390e05f5d2507f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:49.146446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1234/2441] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:48 2022] DELETE /identity/v3/users/d566a1c7f0284fb1b390e05f5d2507f9 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:49.794662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:49.797635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:49.811671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d1a3-9b62-4556-997f-bc55b7a5ef08 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:49.898242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:49.901087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:49.901087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:49.901087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] PATH_INFO: `/v3/projects/860e687f576846cfbfca5dac9a99b067` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:49.901996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=860e687f576846cfbfca5dac9a99b067)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:49.904073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:50.030126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:50.038921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860e687f576846cfbfca5dac9a99b067', 'request_id': 'req-9f627214-1a46-4f1d-b71c-bce654d746b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.133708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860e687f576846cfbfca5dac9a99b067', 'request_id': 'req-9f627214-1a46-4f1d-b71c-bce654d746b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.141629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860e687f576846cfbfca5dac9a99b067', 'request_id': 'req-9f627214-1a46-4f1d-b71c-bce654d746b2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.153679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] The token cache is being invalidate because project 860e687f576846cfbfca5dac9a99b067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:50.153978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.154651 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.156266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1208/2442] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:49 2022] DELETE /identity/v3/projects/860e687f576846cfbfca5dac9a99b067 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:50.160590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:50.162237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:50.177839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ec18-e982-43b6-bce2-8023d66c6ea6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:50.244032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:50.245559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:50.246408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:50.246693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] PATH_INFO: `/v3/users/e27d872fd21e4517958e1550abb706f8` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:50.247943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27d872fd21e4517958e1550abb706f8)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:50.249930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:50.498869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27d872fd21e4517958e1550abb706f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.498869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27d872fd21e4517958e1550abb706f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.509584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27d872fd21e4517958e1550abb706f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.515715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:50.523003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27d872fd21e4517958e1550abb706f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.648119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80ab21eb-ae43-4d76-aabf-3e3874d3e0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27d872fd21e4517958e1550abb706f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:50.655973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1235/2443] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:42:50 2022] DELETE /identity/v3/users/e27d872fd21e4517958e1550abb706f8 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:51.148792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:51.150306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:51.171650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f627214-1a46-4f1d-b71c-bce654d746b2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:51.244531 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:51.245764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:51.245764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:51.245764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] PATH_INFO: `/v3/projects/570de1faf53446ba9a6f27cc937e6eca` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:51.247231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570de1faf53446ba9a6f27cc937e6eca)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:51.249329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:51.395169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570de1faf53446ba9a6f27cc937e6eca', 'request_id': 'req-76823eaf-ec49-4ebe-9088-0337e08cb6f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:51.469912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570de1faf53446ba9a6f27cc937e6eca', 'request_id': 'req-76823eaf-ec49-4ebe-9088-0337e08cb6f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:51.476311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570de1faf53446ba9a6f27cc937e6eca', 'request_id': 'req-76823eaf-ec49-4ebe-9088-0337e08cb6f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:51.489793 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] The token cache is being invalidate because project 570de1faf53446ba9a6f27cc937e6eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:42:51.490090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:51.490759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76823eaf-ec49-4ebe-9088-0337e08cb6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:42:51.493111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1209/2444] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:42:51 2022] DELETE /identity/v3/projects/570de1faf53446ba9a6f27cc937e6eca => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:42:51.498819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa1ab-40af-4675-9f38-55927d69d272 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:51.499329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa1ab-40af-4675-9f38-55927d69d272 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:51.499414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa1ab-40af-4675-9f38-55927d69d272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:51.522367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d2faa1ab-40af-4675-9f38-55927d69d272 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:51.523582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d2faa1ab-40af-4675-9f38-55927d69d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:51.584632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1236/2445] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:42:51.592494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df14d8-070a-4ab3-b844-a8a049d49d04 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:51.592794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df14d8-070a-4ab3-b844-a8a049d49d04 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:51.593070 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df14d8-070a-4ab3-b844-a8a049d49d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:51.612811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-02df14d8-070a-4ab3-b844-a8a049d49d04 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:51.614019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-02df14d8-070a-4ab3-b844-a8a049d49d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:51.622376 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1210/2446] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:51.626679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa1ab-40af-4675-9f38-55927d69d272 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:51.638023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa1ab-40af-4675-9f38-55927d69d272 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:51.642635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:51.643281 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:51.643561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:51.643835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:51.644616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:42:51.644923 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:51.646685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:51.663076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1237/2447] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:42:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:42:51.665187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df14d8-070a-4ab3-b844-a8a049d49d04 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:51.675868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df14d8-070a-4ab3-b844-a8a049d49d04 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:51.680976 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:51.681773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:51.682058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:51.682967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:51.684964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:51.686442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:51.816897 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1211/2448] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:42:51 2022] POST /identity/v3/projects => generated 372 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:42:51.821290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:51.831783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e80ca5-4403-4985-82bd-bec0b7d00a1c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:51.893628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:51.895972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:51.895972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:51.895972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:51.895972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:51.897436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:51.998124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1238/2449] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:42:51 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:42:52.002638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:52.014623 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2182d-93e0-4a39-9155-530846b7c379 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:52.019699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:52.020319 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:52.020635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:52.020911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:52.021663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:52.023250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:52.034357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1212/2450] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:52 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:52.038489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:52.049983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fc24a-8a74-45e0-9ae8-e712bf2d0757 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:52.055037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fabd44-56b8-410c-8500-cbcc7c6fde99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:52.056728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabd44-56b8-410c-8500-cbcc7c6fde99 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:52.057019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabd44-56b8-410c-8500-cbcc7c6fde99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:52.057646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabd44-56b8-410c-8500-cbcc7c6fde99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:52.058610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fabd44-56b8-410c-8500-cbcc7c6fde99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:52.061482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fabd44-56b8-410c-8500-cbcc7c6fde99 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:52.077046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1239/2451] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:42:52 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:42:52.081297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:52.094419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8c417-60e4-4226-9514-e5c2f41a1fc1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:52.100007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:52.102450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:52.102450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:52.102450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] PATH_INFO: `/v3/projects/f06fcba3614a4f428778ccfa4d2ac3eb/users/e2d108f5b3e64655a62cc6d25255ff87/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:52.115652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:52.119201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06fcba3614a4f428778ccfa4d2ac3eb, user_id=e2d108f5b3e64655a62cc6d25255ff87, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:52.122973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:52.225132 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1213/2452] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:42:52 2022] PUT /identity/v3/projects/f06fcba3614a4f428778ccfa4d2ac3eb/users/e2d108f5b3e64655a62cc6d25255ff87/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:42:52.231183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61b17-0394-4ac8-8ff1-bc861ec466cb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:52.231479 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61b17-0394-4ac8-8ff1-bc861ec466cb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:52.231759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61b17-0394-4ac8-8ff1-bc861ec466cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:52.254221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-56a61b17-0394-4ac8-8ff1-bc861ec466cb None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:52.254432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-56a61b17-0394-4ac8-8ff1-bc861ec466cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:52.298872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:42:52.318090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1240/2453] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:52.327643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:52.330048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:52.396151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f700-050a-43a0-b67b-3f181e2d7a2b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:52.400105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bbeef-25fe-4c14-9984-092e76b4985d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:52.401389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bbeef-25fe-4c14-9984-092e76b4985d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:52.401806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bbeef-25fe-4c14-9984-092e76b4985d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:52.402085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bbeef-25fe-4c14-9984-092e76b4985d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:52.414072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bbeef-25fe-4c14-9984-092e76b4985d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:52.415006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bbeef-25fe-4c14-9984-092e76b4985d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:52.426074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1214/2454] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:53.704123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6e3e9-bf20-4bb5-9634-028e9c025876 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:53.704696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6e3e9-bf20-4bb5-9634-028e9c025876 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:53.704696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6e3e9-bf20-4bb5-9634-028e9c025876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:53.725205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c4b6e3e9-bf20-4bb5-9634-028e9c025876 None None] MFA Rules not processed for user `39f0a10b47b84bd1bd8ff8177362d791`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:53.726425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c4b6e3e9-bf20-4bb5-9634-028e9c025876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:53.793021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1241/2455] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:55.584533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3736-87be-4b2d-bcfd-a9b35f372d87 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:55.585066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3736-87be-4b2d-bcfd-a9b35f372d87 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:55.585150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3736-87be-4b2d-bcfd-a9b35f372d87 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:55.588902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1215/2456] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:42:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:42:55.601000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108fdc8-58b0-4e68-86f3-ba10153f66ad None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:55.601430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108fdc8-58b0-4e68-86f3-ba10153f66ad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:55.601887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108fdc8-58b0-4e68-86f3-ba10153f66ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:55.638997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9108fdc8-58b0-4e68-86f3-ba10153f66ad None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:55.640609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9108fdc8-58b0-4e68-86f3-ba10153f66ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:55.748885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1242/2457] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:42:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:55.757125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3736-87be-4b2d-bcfd-a9b35f372d87 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:55.759375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b0cf3736-87be-4b2d-bcfd-a9b35f372d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:55.837899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3736-87be-4b2d-bcfd-a9b35f372d87 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:55.843418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6150f-bff5-4cf7-b1d3-ea7140eb8212 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0207287721f5489f8ac7a258356e57bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:55.844409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6150f-bff5-4cf7-b1d3-ea7140eb8212 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:55.844942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6150f-bff5-4cf7-b1d3-ea7140eb8212 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:55.845456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6150f-bff5-4cf7-b1d3-ea7140eb8212 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:55.862294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6150f-bff5-4cf7-b1d3-ea7140eb8212 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:55.863989 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6150f-bff5-4cf7-b1d3-ea7140eb8212 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:55.887477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1216/2458] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:56.251023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452e3c1-0ed2-4564-ae4c-d8813a924fd5 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:56.251572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452e3c1-0ed2-4564-ae4c-d8813a924fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:56.251976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452e3c1-0ed2-4564-ae4c-d8813a924fd5 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:56.253727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1243/2459] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:42:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:42:56.264286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3743d8-7762-4fe0-b9c3-c72ff2c4191e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:56.266055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3743d8-7762-4fe0-b9c3-c72ff2c4191e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:56.266055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3743d8-7762-4fe0-b9c3-c72ff2c4191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:56.294726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5e3743d8-7762-4fe0-b9c3-c72ff2c4191e None None] MFA Rules not processed for user `c6ecc741022f4d6189f4d3f19b4cdbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:56.295084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5e3743d8-7762-4fe0-b9c3-c72ff2c4191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:56.304760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1217/2460] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:56.323317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452e3c1-0ed2-4564-ae4c-d8813a924fd5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:56.325327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e452e3c1-0ed2-4564-ae4c-d8813a924fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:56.339476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452e3c1-0ed2-4564-ae4c-d8813a924fd5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:56.343376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df601d-6366-4f11-9dfe-f5597a214d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:56.345162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df601d-6366-4f11-9dfe-f5597a214d1f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:56.348548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df601d-6366-4f11-9dfe-f5597a214d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:56.348548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df601d-6366-4f11-9dfe-f5597a214d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:56.361150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df601d-6366-4f11-9dfe-f5597a214d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:56.363271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df601d-6366-4f11-9dfe-f5597a214d1f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:56.378145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1244/2461] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:42:56.484659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c996ac-1f88-497f-a7d8-e7dc2f49be66 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:56.484958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c996ac-1f88-497f-a7d8-e7dc2f49be66 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:56.489730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c996ac-1f88-497f-a7d8-e7dc2f49be66 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:56.489730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1218/2462] 10.222.0.128 () {54 vars in 926 bytes} [Tue Jul 26 15:42:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 15:42:56.494608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb4ab3-b884-4657-8138-39447548a210 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:56.494985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb4ab3-b884-4657-8138-39447548a210 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:56.495178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb4ab3-b884-4657-8138-39447548a210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:56.518587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bbdb4ab3-b884-4657-8138-39447548a210 None None] MFA Rules not processed for user `92e40a33db1a41d59d9a7a3bef5f3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:42:56.522817 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bbdb4ab3-b884-4657-8138-39447548a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:42:56.532734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1245/2463] 10.222.0.128 () {58 vars in 1023 bytes} [Tue Jul 26 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:42:56.543813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c996ac-1f88-497f-a7d8-e7dc2f49be66 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:42:56.578455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c996ac-1f88-497f-a7d8-e7dc2f49be66 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:42:56.588949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:42:56.589979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:42:56.590341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:42:56.590692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:42:56.606920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:42:56.609526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:42:56.647792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1219/2464] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:00.403635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1f6d0-2c2f-44aa-ab97-38baf63312ef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:00.404591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1f6d0-2c2f-44aa-ab97-38baf63312ef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:00.405024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1f6d0-2c2f-44aa-ab97-38baf63312ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:00.487087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d7a1f6d0-2c2f-44aa-ab97-38baf63312ef None None] MFA Rules not processed for user `e2d108f5b3e64655a62cc6d25255ff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:00.492660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d7a1f6d0-2c2f-44aa-ab97-38baf63312ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:00.601752 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:00.622268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1246/2465] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:00.627031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:00.641364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3385846-f8c0-4c05-bb76-27ebb341ace8 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:00.653273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:00.654466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:00.654953 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:00.655431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:00.656332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:00.658543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:00.801150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1f6d0-2c2f-44aa-ab97-38baf63312ef None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:00.803408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1220/2466] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:00 2022] POST /identity/v3/projects => generated 374 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:00.813615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1f6d0-2c2f-44aa-ab97-38baf63312ef None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:00.893734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:00.895244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:00.895244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:00.895244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:00.895927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:00.898950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:01.019723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1247/2467] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:00 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:01.035803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:01.036453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0918367-0c1d-47ec-9550-60d30ca32f10 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:01.044267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:01.045325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:01.045804 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:01.046279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:01.047199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:01.049562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:01.061837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1221/2468] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:01 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:01.073932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:01.087794 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f0ec1-64ea-4473-8200-efee5d7ee78f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:01.101830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:01.109440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:01.109440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:01.109440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] PATH_INFO: `/v3/projects/accee6d6b3bd4180befb4f81ec0d5ca3/users/89ac4c03f5514311832c4de2d9ca464f/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:01.128234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=accee6d6b3bd4180befb4f81ec0d5ca3, user_id=89ac4c03f5514311832c4de2d9ca464f, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:01.132828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:01.295540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1248/2469] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:01 2022] PUT /identity/v3/projects/accee6d6b3bd4180befb4f81ec0d5ca3/users/89ac4c03f5514311832c4de2d9ca464f/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:01.297317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:01.319044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ddb8a-09fa-4e34-b7d1-0d137b217115 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:01.380290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:01.400564 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b07b5-4ab1-48ef-b842-04666d91360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:01.401211 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b07b5-4ab1-48ef-b842-04666d91360c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:01.401496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b07b5-4ab1-48ef-b842-04666d91360c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:01.401773 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b07b5-4ab1-48ef-b842-04666d91360c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:01.405825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b07b5-4ab1-48ef-b842-04666d91360c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:01.409179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b07b5-4ab1-48ef-b842-04666d91360c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:01.426301 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1222/2470] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:01 2022] GET /identity/v3/roles => generated 1427 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:01.427895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:01.459033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e95976-dd83-4a94-91c4-625544afbaa1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:01.483511 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:01.484202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:01.484514 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:01.484810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] PATH_INFO: `/v3/projects/accee6d6b3bd4180befb4f81ec0d5ca3/users/89ac4c03f5514311832c4de2d9ca464f/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:01.497288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=accee6d6b3bd4180befb4f81ec0d5ca3, user_id=89ac4c03f5514311832c4de2d9ca464f, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:01.500043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:01.553616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1249/2471] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:01 2022] PUT /identity/v3/projects/accee6d6b3bd4180befb4f81ec0d5ca3/users/89ac4c03f5514311832c4de2d9ca464f/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:01.578565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de69494-6fcb-4f2c-93cf-2e5c4541ced5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:01.579398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de69494-6fcb-4f2c-93cf-2e5c4541ced5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:01.579830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de69494-6fcb-4f2c-93cf-2e5c4541ced5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:01.626958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4de69494-6fcb-4f2c-93cf-2e5c4541ced5 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:01.630682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4de69494-6fcb-4f2c-93cf-2e5c4541ced5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:01.723778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1223/2472] 10.222.0.128 () {58 vars in 1014 bytes} [Tue Jul 26 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:01.778317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:01.780064 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:01.937986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdfcc7-1baf-4538-a82f-f133498e8ef4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:01.962027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee25689-fb8f-40a8-a6aa-4fc7b3359d5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:01.962753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee25689-fb8f-40a8-a6aa-4fc7b3359d5a service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:01.963179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee25689-fb8f-40a8-a6aa-4fc7b3359d5a service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:01.963465 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee25689-fb8f-40a8-a6aa-4fc7b3359d5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:01.974028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee25689-fb8f-40a8-a6aa-4fc7b3359d5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:01.976456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee25689-fb8f-40a8-a6aa-4fc7b3359d5a service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:01.988384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1250/2473] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:43:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:43:09.083871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de69494-6fcb-4f2c-93cf-2e5c4541ced5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:09.188435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de69494-6fcb-4f2c-93cf-2e5c4541ced5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:09.194609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:09.195936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:09.197180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:09.201222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:09.202845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:09.392161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:09.449465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:09.451542 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:09.484839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1224/2474] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:11.985308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:14.174503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:14.177062 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:14.186575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:14.318790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] MFA Rules not processed for user `89ac4c03f5514311832c4de2d9ca464f`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:14.318790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:14.613439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1251/2475] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:14.737091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:14.738987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:15.011781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acac014-17fe-46b8-86ad-f1b2761cf9d2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:15.018977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:15.019622 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:15.022493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:15.022493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:15.125534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:15.127309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:15.140002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1225/2476] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:18.151841 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:18.154772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:18.179527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b2c01-e85d-42a7-bdb8-501713202170 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:18.189682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:18.200042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:18.200618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:18.202727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] PATH_INFO: `/v3/users/c2e3c15511fe433a9278248c528c86e0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:18.211042 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e3c15511fe433a9278248c528c86e0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:18.216184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:18.486385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e3c15511fe433a9278248c528c86e0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:18.486385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e3c15511fe433a9278248c528c86e0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:18.495206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e3c15511fe433a9278248c528c86e0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:18.521083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e3c15511fe433a9278248c528c86e0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:18.588936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb6c919f-0129-4fd1-8e2f-2dc2bfb1ffda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e3c15511fe433a9278248c528c86e0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:18.605653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1252/2477] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:18 2022] DELETE /identity/v3/users/c2e3c15511fe433a9278248c528c86e0 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:19.054354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:19.076791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983c937-b72c-4f16-af48-7c8b0256f04a service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:19.131419 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:19.199190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:19.200016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:19.200512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:19.200939 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] PATH_INFO: `/v3/projects/f81499626e98449897622291738f07f7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:19.202482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81499626e98449897622291738f07f7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:19.205515 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:19.351214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81499626e98449897622291738f07f7', 'request_id': 'req-24a98918-f882-4c18-ae06-039b29b7ae6d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:19.429061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81499626e98449897622291738f07f7', 'request_id': 'req-24a98918-f882-4c18-ae06-039b29b7ae6d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:19.440194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81499626e98449897622291738f07f7', 'request_id': 'req-24a98918-f882-4c18-ae06-039b29b7ae6d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:19.474310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] The token cache is being invalidate because project f81499626e98449897622291738f07f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:19.474625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:19.475445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-24a98918-f882-4c18-ae06-039b29b7ae6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:19.477518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1226/2478] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:19 2022] DELETE /identity/v3/projects/f81499626e98449897622291738f07f7 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:19.482444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f297cb-dd73-40b1-be7a-7308dd7ecca3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:19.483034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f297cb-dd73-40b1-be7a-7308dd7ecca3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:19.485638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f297cb-dd73-40b1-be7a-7308dd7ecca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:19.606402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-08f297cb-dd73-40b1-be7a-7308dd7ecca3 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:19.608997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-08f297cb-dd73-40b1-be7a-7308dd7ecca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:19.677006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:19.802780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1253/2479] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:19.813054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef369c-e1fd-419d-a5f1-02907a977062 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:19.813396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef369c-e1fd-419d-a5f1-02907a977062 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:19.813606 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef369c-e1fd-419d-a5f1-02907a977062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:19.854538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-76ef369c-e1fd-419d-a5f1-02907a977062 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:19.855787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-76ef369c-e1fd-419d-a5f1-02907a977062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:19.905835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1227/2480] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:19.910176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f297cb-dd73-40b1-be7a-7308dd7ecca3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:19.923565 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f297cb-dd73-40b1-be7a-7308dd7ecca3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:19.938265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14950892-7380-4b8f-a0b1-3f57f0adc4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:19.938869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14950892-7380-4b8f-a0b1-3f57f0adc4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:19.939157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14950892-7380-4b8f-a0b1-3f57f0adc4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:19.939494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14950892-7380-4b8f-a0b1-3f57f0adc4bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:19.940262 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14950892-7380-4b8f-a0b1-3f57f0adc4bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:19.940560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14950892-7380-4b8f-a0b1-3f57f0adc4bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:19.942080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14950892-7380-4b8f-a0b1-3f57f0adc4bf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:19.989242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1254/2481] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:43:19.993827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef369c-e1fd-419d-a5f1-02907a977062 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:20.016800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef369c-e1fd-419d-a5f1-02907a977062 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:20.035366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:20.036218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.036534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.036874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.037585 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:20.041300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:20.133820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.134925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.134925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.159163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1228/2482] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:19 2022] POST /identity/v3/projects => generated 368 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:20.168005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:20.174190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] MFA Rules not processed for user `e2d108f5b3e64655a62cc6d25255ff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:20.175707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:20.311087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca50f-3163-45fd-9f70-404dc0944774 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:20.316573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:20.316972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1255/2483] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:20.317241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.317526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.317802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.318192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:20.318487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:20.319907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:20.320516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:20.425051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1229/2484] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:20 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:20.428862 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:20.442761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e5fdf-4d0e-4a91-bf71-553262c6286a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:20.454624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:20.455550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.455845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.457401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.457401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:20.458634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:20.480286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1230/2485] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:20 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:20.481433 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:20.496300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc670c-97de-4e1f-aa16-4b0d81ec3e0a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:20.499565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ac920-d316-4288-ad71-a589f6afd229 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:20.505743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:20.507304 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.507588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.508121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.510032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:20.511694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:20.512360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9f7f-215a-4fbb-a39d-67708e887f81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:20.514351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:20.515033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9f7f-215a-4fbb-a39d-67708e887f81 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.515560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9f7f-215a-4fbb-a39d-67708e887f81 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.515937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9f7f-215a-4fbb-a39d-67708e887f81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.522367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cb8b9f7f-215a-4fbb-a39d-67708e887f81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:20.529852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1231/2486] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:20 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:20.530858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:20.543602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fb10-03ac-4b1c-8667-80c980cb5afb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:20.550534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:20.551640 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.552682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.553009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] PATH_INFO: `/v3/projects/8d6b478ddf794bdb8eaf540873d828c4/users/7a5d7f1d4abe4d878e83ac6c5d80b996/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.555023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9f7f-215a-4fbb-a39d-67708e887f81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:20.560699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9f7f-215a-4fbb-a39d-67708e887f81 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:20.579645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d6b478ddf794bdb8eaf540873d828c4, user_id=7a5d7f1d4abe4d878e83ac6c5d80b996, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:20.583403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:20.590425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1256/2487] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:20.676305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1232/2488] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:20 2022] PUT /identity/v3/projects/8d6b478ddf794bdb8eaf540873d828c4/users/7a5d7f1d4abe4d878e83ac6c5d80b996/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:20.688857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85807141-fcad-4f5c-9e3d-fa70564007ba None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.689144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85807141-fcad-4f5c-9e3d-fa70564007ba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.690001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85807141-fcad-4f5c-9e3d-fa70564007ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.719388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-85807141-fcad-4f5c-9e3d-fa70564007ba None None] MFA Rules not processed for user `7a5d7f1d4abe4d878e83ac6c5d80b996`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:20.721607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-85807141-fcad-4f5c-9e3d-fa70564007ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:20.741050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:20.807930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1257/2489] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:20.816300 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:20.823467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:20.929110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13414ba3-4a86-480f-b8ad-dfa737f121d8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:20.936866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:20.937800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:20.938949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:20.939740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:20.966571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:20.973374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:20.990752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1233/2490] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:23.596715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:23.598039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:23.598485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:23.652833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] MFA Rules not processed for user `f35a6f8e44fa42dca9ab2ea540ba57ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:23.654063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:23.770789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:23.772544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:23.776796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1258/2491] 10.222.0.128 () {58 vars in 972 bytes} [Tue Jul 26 15:43:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:23.860842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba084b-84dc-4937-a5f3-b5500fbca543 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:23.869916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97295dc4a04143fe879bef062964e4e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:23.871205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:23.871746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:23.874566 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:23.885815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:23.888006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:23.916554 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1234/2492] 10.222.0.128 () {60 vars in 1389 bytes} [Tue Jul 26 15:43:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:43:28.351772 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:28.353810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:28.443253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9cebc-67c9-4590-8c08-2fcdead88ef2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:28.447294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70716d-f8b8-49fc-a6a2-4be51ee9c068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:28.447915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70716d-f8b8-49fc-a6a2-4be51ee9c068 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:28.448201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70716d-f8b8-49fc-a6a2-4be51ee9c068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:28.448508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70716d-f8b8-49fc-a6a2-4be51ee9c068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:28.450311 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9c70716d-f8b8-49fc-a6a2-4be51ee9c068 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:28.504842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:28.520855 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70716d-f8b8-49fc-a6a2-4be51ee9c068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:28.526501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70716d-f8b8-49fc-a6a2-4be51ee9c068 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:28.532305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1259/2493] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:31.333574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:31.347931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08b526-4c1d-4953-9df0-aa5073e883ed service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:31.353330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:31.354247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:31.354569 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:31.354848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] PATH_INFO: `/v3/users/7a5d7f1d4abe4d878e83ac6c5d80b996` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:31.356024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5d7f1d4abe4d878e83ac6c5d80b996)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:31.358033 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:31.521991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5d7f1d4abe4d878e83ac6c5d80b996'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:31.530388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5d7f1d4abe4d878e83ac6c5d80b996'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:31.532799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5d7f1d4abe4d878e83ac6c5d80b996'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:31.535146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:31.593917 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5d7f1d4abe4d878e83ac6c5d80b996'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:31.594590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5d7f1d4abe4d878e83ac6c5d80b996'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:31.615236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1235/2494] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:31 2022] DELETE /identity/v3/users/7a5d7f1d4abe4d878e83ac6c5d80b996 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:31.622287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:31.622823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:31.622909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:31.642898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:31.644182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:31.709489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1260/2495] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:31.721498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:31.723026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:31.786397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e45466-a44d-4e8b-b24a-f0e8e132b215 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:31.790664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:31.791577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:31.792242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:31.792400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:31.807108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:31.808663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:31.820195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1236/2496] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:32.513689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:32.515233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:32.535909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63893dc6-ba5a-4e66-ab1f-450bbb8e9c44 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:32.542176 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:32.542849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:32.543126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:32.544342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] PATH_INFO: `/v3/projects/8d6b478ddf794bdb8eaf540873d828c4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:32.546429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d6b478ddf794bdb8eaf540873d828c4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:32.553919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:32.657877 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:32.659803 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:32.676080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5067537-a74e-42b7-90ff-dc4d46f83aa7 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:32.680984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:32.681632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:32.682241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:32.682530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] PATH_INFO: `/v3/users/494aa7af44284866b02f2a295b8b05aa` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:32.684649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] RBAC: Authorizing `identity:delete_user(user_id=494aa7af44284866b02f2a295b8b05aa)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:32.686604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:32.707721 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d6b478ddf794bdb8eaf540873d828c4', 'request_id': 'req-310ccd12-d0c0-4a73-8aa7-b438d77407d4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.715263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d6b478ddf794bdb8eaf540873d828c4', 'request_id': 'req-310ccd12-d0c0-4a73-8aa7-b438d77407d4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.720973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d6b478ddf794bdb8eaf540873d828c4', 'request_id': 'req-310ccd12-d0c0-4a73-8aa7-b438d77407d4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.822334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] The token cache is being invalidate because project 8d6b478ddf794bdb8eaf540873d828c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:32.822657 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.823348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-310ccd12-d0c0-4a73-8aa7-b438d77407d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.824957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1261/2497] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:32 2022] DELETE /identity/v3/projects/8d6b478ddf794bdb8eaf540873d828c4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:32.829902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08630-777c-4ba5-802b-4304134b7f2d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:32.830198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08630-777c-4ba5-802b-4304134b7f2d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:32.830483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08630-777c-4ba5-802b-4304134b7f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:32.857379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f7f08630-777c-4ba5-802b-4304134b7f2d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:32.858601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f7f08630-777c-4ba5-802b-4304134b7f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:32.882958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '494aa7af44284866b02f2a295b8b05aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.890743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '494aa7af44284866b02f2a295b8b05aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.891568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '494aa7af44284866b02f2a295b8b05aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.921720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1262/2498] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:32.929765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498b9a5-4585-45b5-ad68-c8dbb9d4ed3d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:32.930052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498b9a5-4585-45b5-ad68-c8dbb9d4ed3d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:32.930333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498b9a5-4585-45b5-ad68-c8dbb9d4ed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:32.948425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '494aa7af44284866b02f2a295b8b05aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.949427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '494aa7af44284866b02f2a295b8b05aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:32.951481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7498b9a5-4585-45b5-ad68-c8dbb9d4ed3d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:32.952738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7498b9a5-4585-45b5-ad68-c8dbb9d4ed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:32.961244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1263/2499] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:32.966319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498b9a5-4585-45b5-ad68-c8dbb9d4ed3d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:32.970813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:32.976322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1237/2500] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:32 2022] DELETE /identity/v3/users/494aa7af44284866b02f2a295b8b05aa => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:32.977452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498b9a5-4585-45b5-ad68-c8dbb9d4ed3d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.041742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.042388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.042754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.043013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.043806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:33.044034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.046078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.060655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1264/2501] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:43:33.064692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.077339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c58a44-08d2-4994-938d-8cd991efb21c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.082445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.083065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.083352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.083633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.084336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.086305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.213454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1238/2502] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:33 2022] POST /identity/v3/projects => generated 376 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:33.217996 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.229063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7f0a7-37ea-4f3f-882d-8f19763f9f25 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.245956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.248313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:33.291655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:33.298314 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.298950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.299234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.299510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.300365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.302168 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.324185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b7c7-dfe0-4f1f-90a2-cf0d99328e38 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.328826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.329490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.329780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.330254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] PATH_INFO: `/v3/projects/1ae9a8b5a9ec48f392c43a71f142b102` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.331913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae9a8b5a9ec48f392c43a71f142b102)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.333857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.398874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1265/2503] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:33 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:33.404275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.414347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307bad4-0063-48d1-b02e-d96acc59be8e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.420013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.420656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.420943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.421221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.421917 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.423812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.435166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1266/2504] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:33 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:33.438942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.445969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae9a8b5a9ec48f392c43a71f142b102', 'request_id': 'req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:33.449101 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34b89e-4878-400a-b15a-a9bd1136ee0b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.512700 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.513326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.513699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.513938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.514636 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.516244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.528122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1267/2505] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:33 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:33.532124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.536440 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae9a8b5a9ec48f392c43a71f142b102', 'request_id': 'req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:33.542840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a61fd-618a-40cd-8845-00ba05c00f3b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.545720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae9a8b5a9ec48f392c43a71f142b102', 'request_id': 'req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:33.548403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.549116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.549399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.549674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] PATH_INFO: `/v3/projects/4e23e457e026432eafe2b7f626bef6ca/users/097d146673b5420fa24bec096e05fd11/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.556349 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] The token cache is being invalidate because project 1ae9a8b5a9ec48f392c43a71f142b102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:33.556685 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:33.557537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bab944b-20e6-4c4f-8c4c-4cb66ac9f72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:33.559147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1239/2506] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:33 2022] DELETE /identity/v3/projects/1ae9a8b5a9ec48f392c43a71f142b102 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:33.559616 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:33.564144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8075f-c141-4e32-b18b-b1ae660225a6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.564694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8075f-c141-4e32-b18b-b1ae660225a6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.564982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8075f-c141-4e32-b18b-b1ae660225a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.566910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e23e457e026432eafe2b7f626bef6ca, user_id=097d146673b5420fa24bec096e05fd11, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.570126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.587313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5fb8075f-c141-4e32-b18b-b1ae660225a6 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:33.588716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5fb8075f-c141-4e32-b18b-b1ae660225a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:33.596797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1240/2507] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:33.605169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e84b98-dc62-4d2f-805a-14cf5720bf9d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.605456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e84b98-dc62-4d2f-805a-14cf5720bf9d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.605732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e84b98-dc62-4d2f-805a-14cf5720bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.626286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-96e84b98-dc62-4d2f-805a-14cf5720bf9d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:33.642682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-96e84b98-dc62-4d2f-805a-14cf5720bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:33.642682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1241/2508] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:33.645351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e84b98-dc62-4d2f-805a-14cf5720bf9d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.654345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e84b98-dc62-4d2f-805a-14cf5720bf9d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.658995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb55cc-cc10-4142-9fa1-3b19ae01e994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.659621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb55cc-cc10-4142-9fa1-3b19ae01e994 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.659904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb55cc-cc10-4142-9fa1-3b19ae01e994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.660284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb55cc-cc10-4142-9fa1-3b19ae01e994 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.661078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb55cc-cc10-4142-9fa1-3b19ae01e994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:33.661386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb55cc-cc10-4142-9fa1-3b19ae01e994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.663012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb55cc-cc10-4142-9fa1-3b19ae01e994 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.669128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1268/2509] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:33 2022] PUT /identity/v3/projects/4e23e457e026432eafe2b7f626bef6ca/users/097d146673b5420fa24bec096e05fd11/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:33.678941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1242/2510] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:43:33.680230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.687544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.687764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.688045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.718983 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] MFA Rules not processed for user `097d146673b5420fa24bec096e05fd11`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:33.720209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:33.750505 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b9efe-0ab5-4506-a0cd-cda45279337b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.755256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.755920 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.756165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.756441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.757166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.759916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.788455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1243/2511] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:33.800980 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.802480 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:33.831010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:33.841881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1269/2512] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:33 2022] POST /identity/v3/projects => generated 392 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:33.845982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:33.879395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0f0fe-9d59-487c-a3c8-321afdee2011 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.911686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb83f6-74c5-4ef0-8abb-262c6c09e73e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:33.919319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.919959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.920248 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.920927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.921241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:33.923452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:33.950746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:33.951437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:33.951835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:33.952811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:33.994836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1270/2513] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:33 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:33.998902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.015210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f53e1-ca6d-4b17-b9db-6b782f62e7e2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.018061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.018698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.018986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.019268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.019983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.021148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.021640 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.022858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.033921 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1271/2514] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:33 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:34.037800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1244/2515] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:34.037941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.050428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4d43-719e-42bc-8f4e-b8120d8333f9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.058913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a487c15-280c-4c9a-b0af-287bba8a1f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.059632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487c15-280c-4c9a-b0af-287bba8a1f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.059922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487c15-280c-4c9a-b0af-287bba8a1f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.060333 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487c15-280c-4c9a-b0af-287bba8a1f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.061113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a487c15-280c-4c9a-b0af-287bba8a1f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.063717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a487c15-280c-4c9a-b0af-287bba8a1f95 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.076565 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1272/2516] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:34 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:34.080833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.096325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada9953-cbb5-49e4-9867-d11fb86ba49e service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.101292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.101990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.102280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.102667 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] PATH_INFO: `/v3/projects/f4ce865cbb3f42bdbdafd4767b5e7854/users/6a78f7401ce84c97be87eb72111a2fce/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.123533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ce865cbb3f42bdbdafd4767b5e7854, user_id=6a78f7401ce84c97be87eb72111a2fce, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.127297 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.214502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1245/2517] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:34 2022] PUT /identity/v3/projects/f4ce865cbb3f42bdbdafd4767b5e7854/users/6a78f7401ce84c97be87eb72111a2fce/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:34.224572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c78ff1-df6d-4f34-b893-ff0afb18e720 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.224870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c78ff1-df6d-4f34-b893-ff0afb18e720 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.225159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c78ff1-df6d-4f34-b893-ff0afb18e720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.255031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a5c78ff1-df6d-4f34-b893-ff0afb18e720 None None] MFA Rules not processed for user `6a78f7401ce84c97be87eb72111a2fce`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:34.258376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a5c78ff1-df6d-4f34-b893-ff0afb18e720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:34.276113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.277906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:34.295924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:34.319822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:34.335641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1273/2518] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:34.340655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c78ff1-df6d-4f34-b893-ff0afb18e720 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.348000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b57d17-f6ae-4c41-8b07-42ca7d5e6d9d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.350927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c78ff1-df6d-4f34-b893-ff0afb18e720 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.353818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.354424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.354733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.354992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] PATH_INFO: `/v3/users/097d146673b5420fa24bec096e05fd11` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.356174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=097d146673b5420fa24bec096e05fd11)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.356404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.357255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.357456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.357748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.358230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.358671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.361463 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.475116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1274/2519] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:34 2022] POST /identity/v3/projects => generated 394 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:34.479298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.597460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097d146673b5420fa24bec096e05fd11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:34.605920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097d146673b5420fa24bec096e05fd11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:34.606653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097d146673b5420fa24bec096e05fd11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:34.668590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5697-5ee2-4b20-9b65-c35cd0285b8a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.673547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.674169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.674454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.674734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.675401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.677391 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.679125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097d146673b5420fa24bec096e05fd11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:34.679746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-009a5b6c-6469-4563-ae1e-159e97009afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097d146673b5420fa24bec096e05fd11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:34.699158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1246/2520] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:34 2022] DELETE /identity/v3/users/097d146673b5420fa24bec096e05fd11 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:34.703579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.703869 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.704152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.725087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:34.725087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:34.751787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1275/2521] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:34 2022] POST /identity/v3/users => generated 304 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:34.755759 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.757597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:34.771099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1d95-2e71-4248-a27d-6c12d77ed8b4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.794365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1247/2522] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:34.801377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.803220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:34.847195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.848127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.848691 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.849161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.850063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.853178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.867067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1276/2523] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:34 2022] GET /identity/v3/roles => generated 1427 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:34.879109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970759e8-4830-4b19-b7d0-0c9d585fee35 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.879541 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:34.883101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.884383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.884767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.885125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.899914 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.901614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:34.903560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5432419-5deb-481a-83f7-0be3165c9aa4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:34.906631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:34.909289 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:34.910399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:34.910913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:34.911258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] PATH_INFO: `/v3/projects/639b43f570a94ea4a5242dba30da9fe1/users/cf47c5e06dd24e13a474b05d8881fea5/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:34.914168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1248/2524] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:34.938804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] RBAC: Authorizing `identity:create_grant(project_id=639b43f570a94ea4a5242dba30da9fe1, user_id=cf47c5e06dd24e13a474b05d8881fea5, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:34.941624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.013001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1277/2525] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:34 2022] PUT /identity/v3/projects/639b43f570a94ea4a5242dba30da9fe1/users/cf47c5e06dd24e13a474b05d8881fea5/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:35.017186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.030338 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dc669-ae36-4986-a013-6ad6c81b5f47 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:35.095555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f61342-26bb-4127-835f-dcf9c4715949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:35.096757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f61342-26bb-4127-835f-dcf9c4715949 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.097477 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f61342-26bb-4127-835f-dcf9c4715949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.098182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f61342-26bb-4127-835f-dcf9c4715949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.098974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f61342-26bb-4127-835f-dcf9c4715949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:35.100766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f61342-26bb-4127-835f-dcf9c4715949 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.114002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1249/2526] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:35 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:35.118367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.136186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1771dd-fe54-4e7f-9bc6-ce479c34506d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:35.138932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:35.139671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.140023 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.140492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] PATH_INFO: `/v3/projects/639b43f570a94ea4a5242dba30da9fe1/users/cf47c5e06dd24e13a474b05d8881fea5/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.142986 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=639b43f570a94ea4a5242dba30da9fe1, user_id=cf47c5e06dd24e13a474b05d8881fea5, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:35.146065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.150161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:35.158532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1278/2527] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:35 2022] PUT /identity/v3/projects/639b43f570a94ea4a5242dba30da9fe1/users/cf47c5e06dd24e13a474b05d8881fea5/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:35.170129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.170741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.170978 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.211484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] MFA Rules not processed for user `cf47c5e06dd24e13a474b05d8881fea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:35.214673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:35.294017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1250/2528] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:35.305142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.306963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:35.340755 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.342312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:35.408555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942781-f4d2-4e6a-88a1-09e1de866191 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:35.412266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:35.412919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.413201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.413476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.423972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:35.425158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61057-c87e-4f91-bcc8-a9b67c030a9c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:35.428142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.429993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:35.431771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.432148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.432572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] PATH_INFO: `/v3/projects/4e23e457e026432eafe2b7f626bef6ca` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.433785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e23e457e026432eafe2b7f626bef6ca)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:35.435678 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.436238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1279/2529] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:35 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:35.464815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:35.486431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.487806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:35.499577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367196-7aed-4c1e-bea9-af83cb5f5d7e service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:35.503454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb470eef-d82d-41b0-b11d-f2a81f263a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:35.504092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb470eef-d82d-41b0-b11d-f2a81f263a70 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.504659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb470eef-d82d-41b0-b11d-f2a81f263a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.505020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb470eef-d82d-41b0-b11d-f2a81f263a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.507149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bb470eef-d82d-41b0-b11d-f2a81f263a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:35.538510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e23e457e026432eafe2b7f626bef6ca', 'request_id': 'req-6a348fbd-6e93-4d67-9990-edecace8b2f2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:35.570043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb470eef-d82d-41b0-b11d-f2a81f263a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:35.571561 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb470eef-d82d-41b0-b11d-f2a81f263a70 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.580745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1280/2530] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:35.645142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e23e457e026432eafe2b7f626bef6ca', 'request_id': 'req-6a348fbd-6e93-4d67-9990-edecace8b2f2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:35.652209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e23e457e026432eafe2b7f626bef6ca', 'request_id': 'req-6a348fbd-6e93-4d67-9990-edecace8b2f2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:35.663965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] The token cache is being invalidate because project 4e23e457e026432eafe2b7f626bef6ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:35.664261 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:35.665650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6a348fbd-6e93-4d67-9990-edecace8b2f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:35.667234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1251/2531] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:35 2022] DELETE /identity/v3/projects/4e23e457e026432eafe2b7f626bef6ca => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:35.672452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3223be-d664-44ba-a653-15f636df185c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.672835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3223be-d664-44ba-a653-15f636df185c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.673185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3223be-d664-44ba-a653-15f636df185c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.692831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7d3223be-d664-44ba-a653-15f636df185c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:35.694118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7d3223be-d664-44ba-a653-15f636df185c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:35.756513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1281/2532] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:35.765326 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d1b3-5184-472b-8fc1-79c32c654d80 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.765794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d1b3-5184-472b-8fc1-79c32c654d80 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.766067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d1b3-5184-472b-8fc1-79c32c654d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.787494 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b5c5d1b3-5184-472b-8fc1-79c32c654d80 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:35.788731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b5c5d1b3-5184-472b-8fc1-79c32c654d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:35.798973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1252/2533] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:35.803502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3223be-d664-44ba-a653-15f636df185c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.816655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3223be-d664-44ba-a653-15f636df185c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:35.822857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:35.823423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.823706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.823980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.824710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:35.825017 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:35.826668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.839233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1282/2534] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:43:35.843798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5d1b3-5184-472b-8fc1-79c32c654d80 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.853783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5d1b3-5184-472b-8fc1-79c32c654d80 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:35.858572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:35.859188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:35.859475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:35.859777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:35.860491 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:35.862444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:35.949680 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1253/2535] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:35 2022] POST /identity/v3/projects => generated 372 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:35.954153 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:35.960385 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:35.965502 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97093ce8-fbf6-4164-93fc-3db491ea85b8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.025547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.026165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.026448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.026718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.027384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.029332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.126332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1283/2536] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:35 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:36.130441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:36.148142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9203bc2-f540-4f41-aca4-89f84b6e2512 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.153790 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.154415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.154699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.154981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.155689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.157365 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.167534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1254/2537] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:36 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:36.171490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:36.181894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff131e-937b-4823-bd9a-e205a89ac203 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.186611 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88012bf7-830b-495f-ab46-57e2b8b0b0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.187230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88012bf7-830b-495f-ab46-57e2b8b0b0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.191403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88012bf7-830b-495f-ab46-57e2b8b0b0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.191403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88012bf7-830b-495f-ab46-57e2b8b0b0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.191403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88012bf7-830b-495f-ab46-57e2b8b0b0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.191403 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88012bf7-830b-495f-ab46-57e2b8b0b0a7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.203162 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1284/2538] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:36 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:36.207265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:36.220367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fdcf9-6e5d-4ad1-a0b1-d0bb559871cc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.223634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.224679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.224874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.225153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] PATH_INFO: `/v3/projects/ebb5ea484d8d453cb593c5e8ee52ab68/users/a2b89ff3fc8e45869011a5cc4c22a570/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.241698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb5ea484d8d453cb593c5e8ee52ab68, user_id=a2b89ff3fc8e45869011a5cc4c22a570, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.245416 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.349401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1255/2539] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:36 2022] PUT /identity/v3/projects/ebb5ea484d8d453cb593c5e8ee52ab68/users/a2b89ff3fc8e45869011a5cc4c22a570/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:36.359227 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1137ed8-d8bb-4638-be46-85da2955bbfc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.359519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1137ed8-d8bb-4638-be46-85da2955bbfc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.359800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1137ed8-d8bb-4638-be46-85da2955bbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.388553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d1137ed8-d8bb-4638-be46-85da2955bbfc None None] MFA Rules not processed for user `a2b89ff3fc8e45869011a5cc4c22a570`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:36.389747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d1137ed8-d8bb-4638-be46-85da2955bbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:36.463751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1285/2540] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:36.470110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:36.540208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f1925-a49d-4bca-aa21-655a61b1ceaa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.545439 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.546054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.546735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.546735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.548489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.550406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.657599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1256/2541] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:36 2022] POST /identity/v3/projects => generated 374 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:36.661687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1137ed8-d8bb-4638-be46-85da2955bbfc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:36.674404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1137ed8-d8bb-4638-be46-85da2955bbfc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.730997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.731626 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.731916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.732196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.732918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.734859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.816685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1286/2542] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:36 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:36.820654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:36.832649 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf910d-a4c6-4e3e-9771-2da42ea5a51b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.838592 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.838739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.838961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.839241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.839937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.841588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.853202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1257/2543] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:36 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:36.857843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:36.863226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:36.869280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b6edc-7d94-415e-9be5-dce895f76424 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:36.874507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:36.875294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:36.875574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:36.875850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] PATH_INFO: `/v3/projects/7f405b724fdc4de59bbc6a4e18dde721/users/6d8b1079345f4aab97c2e4fd0498340b/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:36.891653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f405b724fdc4de59bbc6a4e18dde721, user_id=6d8b1079345f4aab97c2e4fd0498340b, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:36.894353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:36.993854 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1287/2544] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:36 2022] PUT /identity/v3/projects/7f405b724fdc4de59bbc6a4e18dde721/users/6d8b1079345f4aab97c2e4fd0498340b/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:36.997913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:37.008016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba2a23-f299-4cdf-bfe0-ed502f49e67b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:37.026533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:37.067309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743f08-36eb-478c-97a9-e060af2bddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:37.067937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743f08-36eb-478c-97a9-e060af2bddc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:37.068483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743f08-36eb-478c-97a9-e060af2bddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:37.069189 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743f08-36eb-478c-97a9-e060af2bddc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:37.069916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743f08-36eb-478c-97a9-e060af2bddc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:37.072509 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743f08-36eb-478c-97a9-e060af2bddc4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:37.082720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1258/2545] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:36 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:37.087055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:37.097959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c1429-6dc5-4c88-b188-a6519fb5a6ea admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:37.103063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:37.103722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:37.104003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:37.104274 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] PATH_INFO: `/v3/projects/7f405b724fdc4de59bbc6a4e18dde721/users/6d8b1079345f4aab97c2e4fd0498340b/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:37.106240 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f405b724fdc4de59bbc6a4e18dde721, user_id=6d8b1079345f4aab97c2e4fd0498340b, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:37.109038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:37.120485 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1288/2546] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:37 2022] PUT /identity/v3/projects/7f405b724fdc4de59bbc6a4e18dde721/users/6d8b1079345f4aab97c2e4fd0498340b/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:37.130814 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31689bf-9b5f-41d4-bc35-1efa5226a50c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:37.131110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31689bf-9b5f-41d4-bc35-1efa5226a50c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:37.131396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31689bf-9b5f-41d4-bc35-1efa5226a50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:37.160335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f31689bf-9b5f-41d4-bc35-1efa5226a50c None None] MFA Rules not processed for user `6d8b1079345f4aab97c2e4fd0498340b`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:37.161591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f31689bf-9b5f-41d4-bc35-1efa5226a50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:37.230561 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1259/2547] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:37.241906 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:37.244419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:37.307020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c9555-81d0-482b-a7a2-7c07bfc725e1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:37.310808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8490f0-75fb-4bf2-84f0-5e65e12d95e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:37.311430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8490f0-75fb-4bf2-84f0-5e65e12d95e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:37.311709 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8490f0-75fb-4bf2-84f0-5e65e12d95e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:37.311983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8490f0-75fb-4bf2-84f0-5e65e12d95e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:37.321849 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8490f0-75fb-4bf2-84f0-5e65e12d95e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:37.323378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8490f0-75fb-4bf2-84f0-5e65e12d95e0 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:37.335397 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1289/2548] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:38.238577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31689bf-9b5f-41d4-bc35-1efa5226a50c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:38.311054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31689bf-9b5f-41d4-bc35-1efa5226a50c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:38.315638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:38.316287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:38.316617 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:38.322023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] PATH_INFO: `/v3/users/a2b89ff3fc8e45869011a5cc4c22a570` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:38.322023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b89ff3fc8e45869011a5cc4c22a570)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:38.322023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:38.547615 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:38.551495 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b89ff3fc8e45869011a5cc4c22a570'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:38.560322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b89ff3fc8e45869011a5cc4c22a570'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:38.568043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b89ff3fc8e45869011a5cc4c22a570'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:38.645469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b89ff3fc8e45869011a5cc4c22a570'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:38.646103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b89ff3fc8e45869011a5cc4c22a570'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:38.673781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1260/2549] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:38 2022] DELETE /identity/v3/users/a2b89ff3fc8e45869011a5cc4c22a570 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:38.673911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:38.673911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:38.673911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:38.694124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:38.701076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:38.772056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1290/2550] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:38.783422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:38.784893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:38.851911 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870df76-0dad-4bf3-b279-a01a135c4415 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:38.860966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:38.860966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:38.860966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:38.861944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:38.872500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:38.873944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:38.884816 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1261/2551] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:39.374159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:39.376747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:39.391798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2a4c9-8195-46be-827b-f9d72a8ce101 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:39.396635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:39.397288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:39.397569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:39.397886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] PATH_INFO: `/v3/projects/ebb5ea484d8d453cb593c5e8ee52ab68` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:39.408584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb5ea484d8d453cb593c5e8ee52ab68)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:39.408584 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:39.549718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb5ea484d8d453cb593c5e8ee52ab68', 'request_id': 'req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:39.556362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb5ea484d8d453cb593c5e8ee52ab68', 'request_id': 'req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:39.563600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb5ea484d8d453cb593c5e8ee52ab68', 'request_id': 'req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:39.563600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:39.643545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] The token cache is being invalidate because project ebb5ea484d8d453cb593c5e8ee52ab68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:39.643764 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:39.644388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:39.645930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1291/2552] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:39 2022] DELETE /identity/v3/projects/ebb5ea484d8d453cb593c5e8ee52ab68 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:39.650016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:39.659628 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901c97-135f-4bc0-9335-e6cbdf635db3 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:39.716009 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:39.716682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:39.716963 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:39.717242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] PATH_INFO: `/v3/users/6d8b1079345f4aab97c2e4fd0498340b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:39.718397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8b1079345f4aab97c2e4fd0498340b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:39.720321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:39.889293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8b1079345f4aab97c2e4fd0498340b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:39.897068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8b1079345f4aab97c2e4fd0498340b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:39.897772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8b1079345f4aab97c2e4fd0498340b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.016036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8b1079345f4aab97c2e4fd0498340b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.016684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8b1079345f4aab97c2e4fd0498340b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.018999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:40.036370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1262/2553] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:39 2022] DELETE /identity/v3/users/6d8b1079345f4aab97c2e4fd0498340b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:40.380762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:40.382253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:40.395637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550b7cb-bb89-42b9-a9ad-08c9bb801af2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:40.460449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:40.462266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:40.462266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:40.462481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] PATH_INFO: `/v3/users/6a78f7401ce84c97be87eb72111a2fce` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:40.463697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a78f7401ce84c97be87eb72111a2fce)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:40.465792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:40.469858 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:40.471487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:40.489117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c7bd0-416d-4365-addf-8c1c6136456f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:40.495278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:40.497151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:40.497443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:40.497840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] PATH_INFO: `/v3/projects/7f405b724fdc4de59bbc6a4e18dde721` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:40.499988 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f405b724fdc4de59bbc6a4e18dde721)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:40.501970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:40.663605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f405b724fdc4de59bbc6a4e18dde721', 'request_id': 'req-6e0e3248-406d-47f7-acbb-d54aaa1304d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.673879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a78f7401ce84c97be87eb72111a2fce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.674542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a78f7401ce84c97be87eb72111a2fce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.687090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a78f7401ce84c97be87eb72111a2fce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.701615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a78f7401ce84c97be87eb72111a2fce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.741429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f405b724fdc4de59bbc6a4e18dde721', 'request_id': 'req-6e0e3248-406d-47f7-acbb-d54aaa1304d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.748845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f405b724fdc4de59bbc6a4e18dde721', 'request_id': 'req-6e0e3248-406d-47f7-acbb-d54aaa1304d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.761117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] The token cache is being invalidate because project 7f405b724fdc4de59bbc6a4e18dde721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:40.761408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.762058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6e0e3248-406d-47f7-acbb-d54aaa1304d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.763569 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1263/2554] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:40 2022] DELETE /identity/v3/projects/7f405b724fdc4de59bbc6a4e18dde721 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:40.768226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf799cb-d79e-4308-b4cc-d6ec02e5d932 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:40.768650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf799cb-d79e-4308-b4cc-d6ec02e5d932 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:40.768932 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf799cb-d79e-4308-b4cc-d6ec02e5d932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:40.790237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eaf799cb-d79e-4308-b4cc-d6ec02e5d932 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:40.792075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eaf799cb-d79e-4308-b4cc-d6ec02e5d932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:40.845085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6b8e0cbb-185e-4ace-a201-e2e18623b48f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a78f7401ce84c97be87eb72111a2fce'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:40.852878 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1292/2555] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:40 2022] DELETE /identity/v3/users/6a78f7401ce84c97be87eb72111a2fce => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:40.856631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1264/2556] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:40.857474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:40.857839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:40.858190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:40.864596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45be84-0846-4113-8356-923f3832ab4e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:40.864904 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45be84-0846-4113-8356-923f3832ab4e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:40.865443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45be84-0846-4113-8356-923f3832ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:40.879516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:40.880818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:40.888144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1d45be84-0846-4113-8356-923f3832ab4e None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:40.889583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d45be84-0846-4113-8356-923f3832ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:40.954553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1293/2557] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:40.964332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1265/2558] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:40.965067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:40.966537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:40.968577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45be84-0846-4113-8356-923f3832ab4e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:40.978909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45be84-0846-4113-8356-923f3832ab4e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:40.983263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:40.988302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:40.989746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:40.990101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:40.990390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:40.991115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:40.991427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:40.993942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:40.995990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:41.014246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1266/2559] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:43:41.017499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.034747 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89f23-b7bb-4e9d-8dcf-30d9236cbe78 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.034998 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e7fb-cd13-449d-92ce-7d533c7dae00 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.039522 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.040321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.040679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.040842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.041027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.041145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.041230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.041347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.042429 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.044031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.055866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.062150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.073939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1294/2560] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:41.127504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1267/2561] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:41 2022] POST /identity/v3/projects => generated 366 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:41.131977 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.162544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470df8d-eec4-45c4-8dc8-f4b9da213613 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.232203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.232769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.233052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.233324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.233997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.235941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.367182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1295/2562] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:41 2022] POST /identity/v3/users => generated 284 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:41.371551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.382669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af11c1c-08ab-4c84-b3d7-83a49bfe468a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.389575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.390310 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.390782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.391112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.391831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.393489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.409644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1268/2563] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:41 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:41.413678 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.425953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d763db4-9a74-435c-aad8-8c272e215932 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.432188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.432863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.433142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.434126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.434267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.435815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.451667 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1296/2564] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:41 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:41.453269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.465982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc61ca-b8ae-450b-b0a5-be2e5615147b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.471427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6224fda-021f-4c51-97f4-153b988f794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.472105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6224fda-021f-4c51-97f4-153b988f794c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.472402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6224fda-021f-4c51-97f4-153b988f794c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.472694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6224fda-021f-4c51-97f4-153b988f794c admin admin] PATH_INFO: `/v3/projects/3a9d185171ef41b8b63700080a13f0f5/users/eb28167b4b1246eb80c213ac0cdad922/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.494736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6224fda-021f-4c51-97f4-153b988f794c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9d185171ef41b8b63700080a13f0f5, user_id=eb28167b4b1246eb80c213ac0cdad922, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.494736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6224fda-021f-4c51-97f4-153b988f794c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.580912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.581056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1269/2565] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:41 2022] PUT /identity/v3/projects/3a9d185171ef41b8b63700080a13f0f5/users/eb28167b4b1246eb80c213ac0cdad922/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:41.590740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:41.599013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b9bc-88fb-48a5-b160-43564bf045a5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.599013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b9bc-88fb-48a5-b160-43564bf045a5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.599013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b9bc-88fb-48a5-b160-43564bf045a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.607261 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46a8b1-dada-42aa-afa9-a57432f20bb4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.632725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-72a4b9bc-88fb-48a5-b160-43564bf045a5 None None] MFA Rules not processed for user `eb28167b4b1246eb80c213ac0cdad922`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:41.634750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-72a4b9bc-88fb-48a5-b160-43564bf045a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:41.689276 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.690346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.690731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.691185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] PATH_INFO: `/v3/projects/f4ce865cbb3f42bdbdafd4767b5e7854` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.692999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ce865cbb3f42bdbdafd4767b5e7854)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.696889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.751387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1270/2566] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:41.755881 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4b9bc-88fb-48a5-b160-43564bf045a5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.768941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4b9bc-88fb-48a5-b160-43564bf045a5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.773834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.775232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.775434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.775681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.776339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.778347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:41.876542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:41.899481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ce865cbb3f42bdbdafd4767b5e7854', 'request_id': 'req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:41.906783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ce865cbb3f42bdbdafd4767b5e7854', 'request_id': 'req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:41.911159 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1271/2567] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:41 2022] POST /identity/v3/projects => generated 366 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:41.913821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ce865cbb3f42bdbdafd4767b5e7854', 'request_id': 'req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:41.914050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:41.981232 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbd1a8-53e1-4680-a0e4-5e8a7f0c0d4d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:41.988065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:41.988761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:41.989046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:41.989323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:41.990000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:41.991967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:42.003616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:42.027947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] The token cache is being invalidate because project f4ce865cbb3f42bdbdafd4767b5e7854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:42.028242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:42.028955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:42.030508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1297/2568] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:41 2022] DELETE /identity/v3/projects/f4ce865cbb3f42bdbdafd4767b5e7854 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:42.035223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:42.037355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:42.108037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500001-bf71-4cfe-bd9a-7c12f14d0db6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:42.112734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:42.113382 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.113670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.113953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] PATH_INFO: `/v3/users/cf47c5e06dd24e13a474b05d8881fea5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.115148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf47c5e06dd24e13a474b05d8881fea5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:42.117131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:42.143299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1272/2569] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:41 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:42.152791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:42.154522 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:42.167003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd773650-4617-4d54-b6cd-7c2e3d43245e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:42.172450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:42.173106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.173393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.173674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.174384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:42.176056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:42.190671 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1273/2570] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:42 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:42.194784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:42.205209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d1253-ce6e-4d2b-b626-97987ba47eb5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:42.210600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:42.211389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.212184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.212499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] PATH_INFO: `/v3/projects/fbfdb515f26646cf9c138ee9a6b17cd4/users/95e622f47f9b4eed8762b2651b7ba08f/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.228077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbfdb515f26646cf9c138ee9a6b17cd4, user_id=95e622f47f9b4eed8762b2651b7ba08f, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:42.230933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:42.259076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf47c5e06dd24e13a474b05d8881fea5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:42.259903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf47c5e06dd24e13a474b05d8881fea5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:42.268939 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf47c5e06dd24e13a474b05d8881fea5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:42.281054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf47c5e06dd24e13a474b05d8881fea5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:42.297378 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1274/2571] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:42 2022] PUT /identity/v3/projects/fbfdb515f26646cf9c138ee9a6b17cd4/users/95e622f47f9b4eed8762b2651b7ba08f/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:42.301530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:42.371375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b115acf-5377-4943-9098-360d07ab17f4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:42.377381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:42.378015 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.378305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.378586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.379299 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:42.380956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:42.387326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f8141938-9c35-4b24-beae-7e68871e2dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf47c5e06dd24e13a474b05d8881fea5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:42.396098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1275/2572] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:42 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:42.397227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:42.397785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1298/2573] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:42 2022] DELETE /identity/v3/users/cf47c5e06dd24e13a474b05d8881fea5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:42.401056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:42.413322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8c284-0fe9-48c8-8221-f553547af73f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:42.483540 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:42.484234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.484550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.484834 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] PATH_INFO: `/v3/projects/fbfdb515f26646cf9c138ee9a6b17cd4/users/95e622f47f9b4eed8762b2651b7ba08f/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.486857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbfdb515f26646cf9c138ee9a6b17cd4, user_id=95e622f47f9b4eed8762b2651b7ba08f, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:42.489705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:42.505069 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1276/2574] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:42 2022] PUT /identity/v3/projects/fbfdb515f26646cf9c138ee9a6b17cd4/users/95e622f47f9b4eed8762b2651b7ba08f/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:42.524454 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.525422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.525707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.558903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] MFA Rules not processed for user `95e622f47f9b4eed8762b2651b7ba08f`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:42.560211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:42.594797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:42.642266 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1299/2575] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:42.655535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:42.657901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:42.733202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:42.737230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfab39-4854-4c75-8d90-15712477f353 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:42.742276 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:42.746822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.746822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.746822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.755248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:42.756788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:42.770006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1277/2576] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:42.887429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:42.893614 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:42.984344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3634be-d1fb-4af5-89ff-357385fac9d0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:42.988999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:42.989643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:42.989926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:42.990207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] PATH_INFO: `/v3/projects/639b43f570a94ea4a5242dba30da9fe1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:42.991361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] RBAC: Authorizing `identity:delete_project(project_id=639b43f570a94ea4a5242dba30da9fe1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:42.994211 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.064324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:43.074269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ccd63-1860-4cca-a7b7-b49449f0da18 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:43.078026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475366d9-df24-4f8c-a052-076b383c2bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:43.078634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475366d9-df24-4f8c-a052-076b383c2bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.078918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475366d9-df24-4f8c-a052-076b383c2bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.081099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475366d9-df24-4f8c-a052-076b383c2bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.081099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-475366d9-df24-4f8c-a052-076b383c2bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:43.122681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639b43f570a94ea4a5242dba30da9fe1', 'request_id': 'req-ba03d785-dd58-4529-878a-b0c77a7d717b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:43.129170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639b43f570a94ea4a5242dba30da9fe1', 'request_id': 'req-ba03d785-dd58-4529-878a-b0c77a7d717b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:43.136745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639b43f570a94ea4a5242dba30da9fe1', 'request_id': 'req-ba03d785-dd58-4529-878a-b0c77a7d717b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:43.154580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475366d9-df24-4f8c-a052-076b383c2bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:43.156078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475366d9-df24-4f8c-a052-076b383c2bf8 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.168113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1278/2577] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:43.250253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] The token cache is being invalidate because project 639b43f570a94ea4a5242dba30da9fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:43.250550 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:43.251222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ba03d785-dd58-4529-878a-b0c77a7d717b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:43.252815 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1300/2578] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:42 2022] DELETE /identity/v3/projects/639b43f570a94ea4a5242dba30da9fe1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:43.257733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbb935-b1ed-465b-8fbe-580d217c72ef None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.258125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbb935-b1ed-465b-8fbe-580d217c72ef None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.259362 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbb935-b1ed-465b-8fbe-580d217c72ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.290081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-19bbb935-b1ed-465b-8fbe-580d217c72ef None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:43.290081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-19bbb935-b1ed-465b-8fbe-580d217c72ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:43.379928 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1279/2579] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:43.388976 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b112-573d-41a3-bfb4-88d1666b95f8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.389245 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b112-573d-41a3-bfb4-88d1666b95f8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.389527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b112-573d-41a3-bfb4-88d1666b95f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.417316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3df6b112-573d-41a3-bfb4-88d1666b95f8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:43.418710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3df6b112-573d-41a3-bfb4-88d1666b95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:43.428290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1301/2580] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:43.433487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbb935-b1ed-465b-8fbe-580d217c72ef None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:43.445381 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbb935-b1ed-465b-8fbe-580d217c72ef None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:43.452072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:43.452887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.453171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.453449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.454166 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:43.454468 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:43.456169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.474693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1280/2581] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:43:43.480006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6b112-573d-41a3-bfb4-88d1666b95f8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:43.496376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6b112-573d-41a3-bfb4-88d1666b95f8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:43.502277 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:43.502912 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.503199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.504082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.504585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:43.506348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.596652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1302/2582] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:43 2022] POST /identity/v3/projects => generated 388 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:43.600314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:43.612635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72509-87f8-435d-ad36-33c9ee05a89d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:43.682348 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:43.682990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.683285 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.683567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.684307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:43.686431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.810710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1281/2583] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:43 2022] POST /identity/v3/users => generated 295 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:43.814597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:43.825956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e94d-fee8-458f-82d5-22865edc7377 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:43.831299 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:43.831926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.832208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.832510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.833230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:43.834851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.847183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1303/2584] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:43 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:43.851224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:43.864995 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc88baf-23fb-4160-a88d-5b07243ddeb5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:43.870094 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6aff8-9e1e-4cf8-ab1c-2ccd6e22766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:43.870726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6aff8-9e1e-4cf8-ab1c-2ccd6e22766b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.871104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6aff8-9e1e-4cf8-ab1c-2ccd6e22766b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.871383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6aff8-9e1e-4cf8-ab1c-2ccd6e22766b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.872093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6aff8-9e1e-4cf8-ab1c-2ccd6e22766b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:43.873775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6aff8-9e1e-4cf8-ab1c-2ccd6e22766b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.875286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:43.886815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1282/2585] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:43 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:43.891010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:43.902472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0e2d-21fa-4891-adff-5c669ed83fde admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:43.909000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:43.909738 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:43.910028 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:43.910307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] PATH_INFO: `/v3/projects/4b966d1db71f4dbf97adb148dcc1b974/users/32f239be53714d99b3f9e50dcfc5253c/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:43.928851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b966d1db71f4dbf97adb148dcc1b974, user_id=32f239be53714d99b3f9e50dcfc5253c, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:43.932092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:43.935751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:44.044915 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1304/2586] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:43 2022] PUT /identity/v3/projects/4b966d1db71f4dbf97adb148dcc1b974/users/32f239be53714d99b3f9e50dcfc5253c/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:44.059395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f60f-0ca8-45ec-bf94-d6a0e02eba9f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:44.059709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f60f-0ca8-45ec-bf94-d6a0e02eba9f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:44.060055 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f60f-0ca8-45ec-bf94-d6a0e02eba9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:44.091994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-aea1f60f-0ca8-45ec-bf94-d6a0e02eba9f None None] MFA Rules not processed for user `32f239be53714d99b3f9e50dcfc5253c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:44.093342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-aea1f60f-0ca8-45ec-bf94-d6a0e02eba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:44.197240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1283/2587] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:44.206262 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:44.280797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e3320-e138-4cf8-98bc-6d985f0fcc2b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:44.288238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:44.288924 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:44.289801 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:44.290421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:44.290421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:44.293226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:44.407304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1305/2588] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:44 2022] POST /identity/v3/projects => generated 388 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:44.412629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f60f-0ca8-45ec-bf94-d6a0e02eba9f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:44.430204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f60f-0ca8-45ec-bf94-d6a0e02eba9f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:44.513504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:44.514320 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:44.514706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:44.515092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:44.515886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:44.517987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:44.652138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1284/2589] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:44 2022] POST /identity/v3/users => generated 301 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:44.656435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:44.673193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9738ed-88f5-4705-8489-8c24cfc71c6c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:44.679430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:44.680209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:44.680804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:44.680804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:44.681372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:44.683221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:44.703580 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1306/2590] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:44 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:44.707762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:44.729059 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bc05a-f287-4769-acd3-27bbe59ef267 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:44.739620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:44.740545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:44.740984 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:44.746089 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] PATH_INFO: `/v3/projects/f596823a626140e7a9f408eb7b643bad/users/cb5ec8855cac4a53a3311b5eb3fb3f63/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:44.762854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f596823a626140e7a9f408eb7b643bad, user_id=cb5ec8855cac4a53a3311b5eb3fb3f63, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:44.768351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:44.886370 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1285/2591] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:44 2022] PUT /identity/v3/projects/f596823a626140e7a9f408eb7b643bad/users/cb5ec8855cac4a53a3311b5eb3fb3f63/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:44.889953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:44.931230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5799df-6b97-47f0-9838-63d1764ffdfc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:45.030473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b166bb8-c21a-44be-8819-e314c6082cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:45.031214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b166bb8-c21a-44be-8819-e314c6082cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:45.031569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b166bb8-c21a-44be-8819-e314c6082cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:45.033717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b166bb8-c21a-44be-8819-e314c6082cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:45.033717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b166bb8-c21a-44be-8819-e314c6082cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:45.034494 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b166bb8-c21a-44be-8819-e314c6082cdf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:45.068173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1307/2592] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:44 2022] GET /identity/v3/roles => generated 1427 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:45.073550 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:45.089031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03b7e3-57ec-47e2-b39d-f39ef23f6644 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:45.103745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c380-370e-4724-b155-767a06455337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:45.104449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c380-370e-4724-b155-767a06455337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:45.104760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c380-370e-4724-b155-767a06455337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:45.105041 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c380-370e-4724-b155-767a06455337 admin admin] PATH_INFO: `/v3/projects/f596823a626140e7a9f408eb7b643bad/users/cb5ec8855cac4a53a3311b5eb3fb3f63/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:45.108351 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c380-370e-4724-b155-767a06455337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f596823a626140e7a9f408eb7b643bad, user_id=cb5ec8855cac4a53a3311b5eb3fb3f63, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:45.111461 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c380-370e-4724-b155-767a06455337 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:45.131379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1286/2593] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:45 2022] PUT /identity/v3/projects/f596823a626140e7a9f408eb7b643bad/users/cb5ec8855cac4a53a3311b5eb3fb3f63/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:45.140618 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd98e3-5c31-4fdb-9210-4ff546bd1c45 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:45.140911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd98e3-5c31-4fdb-9210-4ff546bd1c45 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:45.141193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd98e3-5c31-4fdb-9210-4ff546bd1c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:45.176990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a5cd98e3-5c31-4fdb-9210-4ff546bd1c45 None None] MFA Rules not processed for user `cb5ec8855cac4a53a3311b5eb3fb3f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:45.178401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a5cd98e3-5c31-4fdb-9210-4ff546bd1c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:45.271737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:45.312367 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1308/2594] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:45.329307 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c380-370e-4724-b155-767a06455337 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:45.331111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9380c380-370e-4724-b155-767a06455337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:45.390251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:45.411098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c380-370e-4724-b155-767a06455337 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:45.415751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c566f0-4aa9-416b-a03c-c9a336e8a45b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:45.416396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c566f0-4aa9-416b-a03c-c9a336e8a45b service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:45.416716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c566f0-4aa9-416b-a03c-c9a336e8a45b service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:45.416993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c566f0-4aa9-416b-a03c-c9a336e8a45b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:45.428434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c566f0-4aa9-416b-a03c-c9a336e8a45b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:45.430746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c566f0-4aa9-416b-a03c-c9a336e8a45b service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:45.443011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1287/2595] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:45.619085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd98e3-5c31-4fdb-9210-4ff546bd1c45 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:45.694552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd98e3-5c31-4fdb-9210-4ff546bd1c45 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:45.700831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:45.701486 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:45.701768 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:45.702048 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] PATH_INFO: `/v3/users/32f239be53714d99b3f9e50dcfc5253c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:45.703268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f239be53714d99b3f9e50dcfc5253c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:45.705238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:45.939895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f239be53714d99b3f9e50dcfc5253c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:45.940576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f239be53714d99b3f9e50dcfc5253c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:45.947630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f239be53714d99b3f9e50dcfc5253c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:45.962090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f239be53714d99b3f9e50dcfc5253c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:46.049605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f239be53714d99b3f9e50dcfc5253c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:46.057807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1309/2596] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:45 2022] DELETE /identity/v3/users/32f239be53714d99b3f9e50dcfc5253c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:46.062541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:46.063044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:46.063142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:46.082500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:46.083735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:46.151098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1288/2597] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:46.162058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:46.163666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:46.226673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a9fd7-ed7a-4336-b992-9d81eb81c422 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:46.232361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:46.233143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:46.233422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:46.235124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:46.245659 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:46.248947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:46.259098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1310/2598] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:46.834948 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:46.837388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:46.850223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81c873-7121-4ec4-880e-3dd5f8edc99d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:46.857776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:46.858508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:46.858792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:46.859067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] PATH_INFO: `/v3/projects/4b966d1db71f4dbf97adb148dcc1b974` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:46.860328 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b966d1db71f4dbf97adb148dcc1b974)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:46.862614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:46.991352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:46.993473 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:47.006126 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63262010-6f07-4471-af8f-da91246aaa0d service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:47.011681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:47.012362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:47.012808 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:47.013195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] PATH_INFO: `/v3/users/eb28167b4b1246eb80c213ac0cdad922` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:47.014647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb28167b4b1246eb80c213ac0cdad922)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:47.017220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:47.022428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:47.024191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b966d1db71f4dbf97adb148dcc1b974', 'request_id': 'req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.095631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b966d1db71f4dbf97adb148dcc1b974', 'request_id': 'req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.102539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b966d1db71f4dbf97adb148dcc1b974', 'request_id': 'req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.114093 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] The token cache is being invalidate because project 4b966d1db71f4dbf97adb148dcc1b974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:47.114394 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.115044 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.116581 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1289/2599] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:46 2022] DELETE /identity/v3/projects/4b966d1db71f4dbf97adb148dcc1b974 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:47.120389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:47.121737 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:47.167383 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb28167b4b1246eb80c213ac0cdad922'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.168111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb28167b4b1246eb80c213ac0cdad922'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.176380 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb28167b4b1246eb80c213ac0cdad922'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.188941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb28167b4b1246eb80c213ac0cdad922'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.190113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984b4cc-9e3f-40c1-a59c-7ac0e1e8a4ba admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:47.196229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:47.196922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:47.197202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:47.197474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] PATH_INFO: `/v3/users/cb5ec8855cac4a53a3311b5eb3fb3f63` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:47.198725 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5ec8855cac4a53a3311b5eb3fb3f63)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:47.201098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:47.205916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:47.244674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e818326d-49ef-4136-829f-2746bce4e131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb28167b4b1246eb80c213ac0cdad922'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.251175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1311/2600] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:46 2022] DELETE /identity/v3/users/eb28167b4b1246eb80c213ac0cdad922 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:47.255630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:47.255916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:47.256194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:47.278181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:47.279529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:47.350257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1312/2601] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:47.360417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:47.361956 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:47.374644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5ec8855cac4a53a3311b5eb3fb3f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.385892 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5ec8855cac4a53a3311b5eb3fb3f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.386661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5ec8855cac4a53a3311b5eb3fb3f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.418553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38acaa-4dd3-44f2-9cb9-ee38cfd1a957 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:47.424345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:47.425313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:47.425313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:47.427301 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:47.428371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:47.442177 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:47.443625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:47.455999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1313/2602] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:47.472567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5ec8855cac4a53a3311b5eb3fb3f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.473246 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5ec8855cac4a53a3311b5eb3fb3f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:47.495857 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1290/2603] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:47 2022] DELETE /identity/v3/users/cb5ec8855cac4a53a3311b5eb3fb3f63 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:48.050819 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:48.051003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:48.052254 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:48.052926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:48.065319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5db6b-fa04-453c-898e-b40956851a4f service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:48.102597 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:48.143389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636bcbe-dcae-48ab-a7d4-b8cab777ff33 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:48.144954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:48.145607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:48.145890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:48.146169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] PATH_INFO: `/v3/projects/3a9d185171ef41b8b63700080a13f0f5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:48.147322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9d185171ef41b8b63700080a13f0f5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:48.148090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:48.148942 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:48.149350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:48.149495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:48.149759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] PATH_INFO: `/v3/projects/f596823a626140e7a9f408eb7b643bad` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:48.150997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f596823a626140e7a9f408eb7b643bad)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:48.152967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:48.287399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9d185171ef41b8b63700080a13f0f5', 'request_id': 'req-92fb36ae-6293-42c3-9ae6-b852a673c1b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.295571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9d185171ef41b8b63700080a13f0f5', 'request_id': 'req-92fb36ae-6293-42c3-9ae6-b852a673c1b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.302204 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9d185171ef41b8b63700080a13f0f5', 'request_id': 'req-92fb36ae-6293-42c3-9ae6-b852a673c1b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.316479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f596823a626140e7a9f408eb7b643bad', 'request_id': 'req-9ecc98c1-3954-433f-9d0b-bd234b8c098f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.398208 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] The token cache is being invalidate because project 3a9d185171ef41b8b63700080a13f0f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:48.399174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.399689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.401249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1314/2604] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:48 2022] DELETE /identity/v3/projects/3a9d185171ef41b8b63700080a13f0f5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:48.405142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:48.406754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:48.466716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f596823a626140e7a9f408eb7b643bad', 'request_id': 'req-9ecc98c1-3954-433f-9d0b-bd234b8c098f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.473627 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f596823a626140e7a9f408eb7b643bad', 'request_id': 'req-9ecc98c1-3954-433f-9d0b-bd234b8c098f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.474303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb36ae-6293-42c3-9ae6-b852a673c1b0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:48.486418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] The token cache is being invalidate because project f596823a626140e7a9f408eb7b643bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:48.486719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.487427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ecc98c1-3954-433f-9d0b-bd234b8c098f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.488996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1291/2605] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:48 2022] DELETE /identity/v3/projects/f596823a626140e7a9f408eb7b643bad => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:48.494438 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad909-dc03-41e2-92a9-fda7e8a37f90 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:48.494730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad909-dc03-41e2-92a9-fda7e8a37f90 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:48.495043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad909-dc03-41e2-92a9-fda7e8a37f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:48.515103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e1cad909-dc03-41e2-92a9-fda7e8a37f90 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:48.516373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1cad909-dc03-41e2-92a9-fda7e8a37f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:48.525660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:48.539465 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:48.543496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:48.543496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:48.543496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] PATH_INFO: `/v3/users/95e622f47f9b4eed8762b2651b7ba08f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:48.543496 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95e622f47f9b4eed8762b2651b7ba08f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:48.543905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:48.549205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:48.578424 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1292/2606] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:48.587985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc7e6-2113-473e-b10b-026e46868b64 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:48.588280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc7e6-2113-473e-b10b-026e46868b64 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:48.588596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc7e6-2113-473e-b10b-026e46868b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:48.610323 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b74dc7e6-2113-473e-b10b-026e46868b64 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:48.611522 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b74dc7e6-2113-473e-b10b-026e46868b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:48.620782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1293/2607] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:48.624968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dc7e6-2113-473e-b10b-026e46868b64 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:48.634387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dc7e6-2113-473e-b10b-026e46868b64 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:48.639545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:48.640172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:48.640453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:48.640772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:48.641475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:48.641785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:48.643458 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:48.656188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1294/2608] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:43:48.660918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:48.672350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050ad2-e5e6-4f6b-8a0b-fedf88080982 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:48.676977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:48.677611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:48.677903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:48.678183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:48.678843 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:48.681705 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:48.739250 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95e622f47f9b4eed8762b2651b7ba08f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.739925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95e622f47f9b4eed8762b2651b7ba08f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.748143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95e622f47f9b4eed8762b2651b7ba08f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.761464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95e622f47f9b4eed8762b2651b7ba08f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.829578 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1295/2609] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:48 2022] POST /identity/v3/projects => generated 388 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:48.833584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:48.849870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95e622f47f9b4eed8762b2651b7ba08f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:48.858056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1315/2610] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:48 2022] DELETE /identity/v3/users/95e622f47f9b4eed8762b2651b7ba08f => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:48.915596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c825694-db07-4c56-a178-9e7ddfc6961e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.001266 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.001907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.002287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.002571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.003263 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.005414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.121314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1296/2611] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:48 2022] POST /identity/v3/users => generated 295 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:49.125348 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.127414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:49.141339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8c8c6-98ff-4987-bb6f-e55c55cdb883 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.147422 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.148161 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.149233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.149710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.150430 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.152085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.169705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1316/2612] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:49 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:49.170098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.180747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc7bf4-07f8-4010-b63a-9f14b76f3fcc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.186512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.187102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.187387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.187659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.188393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.190021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.192880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:49.204986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1297/2613] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:49 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:49.206192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.218192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f2857-9f5e-4b11-9693-5e8c95b35a2a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.223481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e4bd-e4b1-4a69-9f21-2362d5824afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.223869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e4bd-e4b1-4a69-9f21-2362d5824afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.224159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e4bd-e4b1-4a69-9f21-2362d5824afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.224433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e4bd-e4b1-4a69-9f21-2362d5824afc admin admin] PATH_INFO: `/v3/projects/c707fd49a10e4806a8e33440be5c9f37/users/275ec79bbf594b9b8c9245d37c38ed64/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.241943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e4bd-e4b1-4a69-9f21-2362d5824afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c707fd49a10e4806a8e33440be5c9f37, user_id=275ec79bbf594b9b8c9245d37c38ed64, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.247267 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e4bd-e4b1-4a69-9f21-2362d5824afc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.368257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.370765 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:49.373313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1317/2614] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:49 2022] PUT /identity/v3/projects/c707fd49a10e4806a8e33440be5c9f37/users/275ec79bbf594b9b8c9245d37c38ed64/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:49.384676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195f8a-78c2-4a49-857a-8cc5a12ee7ea None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.385019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195f8a-78c2-4a49-857a-8cc5a12ee7ea None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.385273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195f8a-78c2-4a49-857a-8cc5a12ee7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.389636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee5197-9a5c-44c5-9a90-32f70e09b700 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.417571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8b195f8a-78c2-4a49-857a-8cc5a12ee7ea None None] MFA Rules not processed for user `275ec79bbf594b9b8c9245d37c38ed64`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:49.419141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8b195f8a-78c2-4a49-857a-8cc5a12ee7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:49.468084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.468768 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.469061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.469336 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] PATH_INFO: `/v3/projects/fbfdb515f26646cf9c138ee9a6b17cd4` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.470541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbfdb515f26646cf9c138ee9a6b17cd4)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.472514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.494745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1318/2615] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:49.499226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195f8a-78c2-4a49-857a-8cc5a12ee7ea None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.510542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195f8a-78c2-4a49-857a-8cc5a12ee7ea None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.515451 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.516069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.516350 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.516656 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.517318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.519222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.593430 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbfdb515f26646cf9c138ee9a6b17cd4', 'request_id': 'req-4e58e8f4-afe4-4c34-bf32-9b2508aba308'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:49.640369 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1319/2616] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:49 2022] POST /identity/v3/projects => generated 388 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:49.644443 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.660658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbfdb515f26646cf9c138ee9a6b17cd4', 'request_id': 'req-4e58e8f4-afe4-4c34-bf32-9b2508aba308'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:49.662664 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:49.667521 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbfdb515f26646cf9c138ee9a6b17cd4', 'request_id': 'req-4e58e8f4-afe4-4c34-bf32-9b2508aba308'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:49.680791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] The token cache is being invalidate because project fbfdb515f26646cf9c138ee9a6b17cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:49.681095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:49.681802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4e58e8f4-afe4-4c34-bf32-9b2508aba308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:49.684171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1298/2617] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:49 2022] DELETE /identity/v3/projects/fbfdb515f26646cf9c138ee9a6b17cd4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:49.689929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aa7d1-373a-4af7-ba73-85ba25620910 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.689929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aa7d1-373a-4af7-ba73-85ba25620910 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.689929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aa7d1-373a-4af7-ba73-85ba25620910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.710357 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-974aa7d1-373a-4af7-ba73-85ba25620910 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:49.712605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-974aa7d1-373a-4af7-ba73-85ba25620910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:49.720506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1299/2618] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:49.720686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b48104-f28f-4cbf-bea7-766e87c45195 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.726449 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.727198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.727489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.727767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.728386 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cc1ea-a43c-4d47-b623-d3a0ee77f3a6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.728526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.728711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cc1ea-a43c-4d47-b623-d3a0ee77f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.728990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cc1ea-a43c-4d47-b623-d3a0ee77f3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.730411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.749020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1a1cc1ea-a43c-4d47-b623-d3a0ee77f3a6 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:49.750229 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a1cc1ea-a43c-4d47-b623-d3a0ee77f3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:49.758993 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1300/2619] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:49.763980 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cc1ea-a43c-4d47-b623-d3a0ee77f3a6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.773970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cc1ea-a43c-4d47-b623-d3a0ee77f3a6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.778754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.779415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.779702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.779986 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.780717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:49.781031 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.782634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.794611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1301/2620] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:43:49.799150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.809375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418f165-423e-4c84-a3a8-fa1a25f36137 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.811000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1320/2621] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:49 2022] POST /identity/v3/users => generated 301 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:49.814156 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.814822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.815103 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.815383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.815524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.816081 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.817237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:49.818026 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.830672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e23f5b-7b78-4e1d-9417-c8ae97c1f847 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.837022 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.837850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.838184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.838493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.839444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.841251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.853038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1321/2622] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:49 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:49.857259 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.866693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f77bc1-9ad6-400a-ba0f-9f086bd7b333 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.872712 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.873402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.873688 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.875980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] PATH_INFO: `/v3/projects/555781c028514feb9a76a31fa0b3bbab/users/c90f4a8449314c9fada85f8a050f03d2/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.890358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=555781c028514feb9a76a31fa0b3bbab, user_id=c90f4a8449314c9fada85f8a050f03d2, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.893038 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:49.903563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1302/2623] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:49 2022] POST /identity/v3/projects => generated 384 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:49.907614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:49.961373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:49.981088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b8482-e965-44e2-8102-d211be2902a0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:49.985925 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:49.986552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:49.986835 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:49.987110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:49.987797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:49.989740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.007622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1322/2624] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:49 2022] PUT /identity/v3/projects/555781c028514feb9a76a31fa0b3bbab/users/c90f4a8449314c9fada85f8a050f03d2/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:50.014871 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.082263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9524ad-4278-41fe-a0c6-d235702340a1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.087091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.087753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.088096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.088304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.089041 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.090627 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.101205 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1323/2625] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:50 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:50.105865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.115798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981f3b-1c6a-436a-9790-3a3a68f1ab8e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.122104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1303/2626] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:49 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:50.122800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84022a8-ef1f-494d-849c-6f120e330abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.122800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84022a8-ef1f-494d-849c-6f120e330abb admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.122800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84022a8-ef1f-494d-849c-6f120e330abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.123084 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84022a8-ef1f-494d-849c-6f120e330abb admin admin] PATH_INFO: `/v3/projects/555781c028514feb9a76a31fa0b3bbab/users/c90f4a8449314c9fada85f8a050f03d2/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.124745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.125007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84022a8-ef1f-494d-849c-6f120e330abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=555781c028514feb9a76a31fa0b3bbab, user_id=c90f4a8449314c9fada85f8a050f03d2, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.127680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84022a8-ef1f-494d-849c-6f120e330abb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.136871 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8af6ba-2543-4abd-878a-ac154e30bc30 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.139548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1324/2627] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:50 2022] PUT /identity/v3/projects/555781c028514feb9a76a31fa0b3bbab/users/c90f4a8449314c9fada85f8a050f03d2/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:50.151370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.151670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.151985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.191975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] MFA Rules not processed for user `c90f4a8449314c9fada85f8a050f03d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:50.191975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:50.214652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.215354 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.216054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.216344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.217095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.218707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.234535 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1304/2628] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:50 2022] GET /identity/v3/roles => generated 1427 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:50.236248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.241152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:50.248207 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8f363-b32e-44bb-a820-35f25a31d083 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.264193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.264990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.265277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.265558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.266284 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.267921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.277158 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1325/2629] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:50.286257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1305/2630] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:50 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:43:50.287716 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.288865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:50.294756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.307216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd0bc5-d695-4683-ba25-bf0df92e076f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.316305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.317040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.317322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.317598 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] PATH_INFO: `/v3/projects/444cb09d8b3f4d2db96274f984b31c97/users/a9258de001484e7b9d512f3d79dced2d/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.333886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444cb09d8b3f4d2db96274f984b31c97, user_id=a9258de001484e7b9d512f3d79dced2d, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.336744 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.365571 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b03f6-9cab-476a-b090-3d423fded4a6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.370402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85284127-c7cd-49c2-94f7-23bda45b3639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.371034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85284127-c7cd-49c2-94f7-23bda45b3639 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.371361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85284127-c7cd-49c2-94f7-23bda45b3639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.371837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85284127-c7cd-49c2-94f7-23bda45b3639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.384237 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85284127-c7cd-49c2-94f7-23bda45b3639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.386548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85284127-c7cd-49c2-94f7-23bda45b3639 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.395438 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1326/2631] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:50.413283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1306/2632] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:50 2022] PUT /identity/v3/projects/444cb09d8b3f4d2db96274f984b31c97/users/a9258de001484e7b9d512f3d79dced2d/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:50.425435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c8e88-8ad2-46da-b35d-b834620a18b9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.425865 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c8e88-8ad2-46da-b35d-b834620a18b9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.426287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c8e88-8ad2-46da-b35d-b834620a18b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.458148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d53c8e88-8ad2-46da-b35d-b834620a18b9 None None] MFA Rules not processed for user `a9258de001484e7b9d512f3d79dced2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:50.459368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d53c8e88-8ad2-46da-b35d-b834620a18b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:50.534602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1327/2633] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:50.539761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.605848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f7258-b83c-4a7c-ab8e-92e60999d736 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.610918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.611539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.611825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.612131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.612833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.614791 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.704442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1307/2634] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:50 2022] POST /identity/v3/projects => generated 388 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:50.708567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c8e88-8ad2-46da-b35d-b834620a18b9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.721444 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c8e88-8ad2-46da-b35d-b834620a18b9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.783145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.783914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.784321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.784722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.785828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.790095 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.868978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:50.874381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1328/2635] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:50 2022] POST /identity/v3/users => generated 301 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:50.878644 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.890699 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a514c5a-8ea1-4ffd-a37e-3d09ae1462a7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.895380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.896021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.896306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.896772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.897493 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.899721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:50.911612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1308/2636] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:50 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:50.915909 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:50.926549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811b8b-99c0-4072-abe3-4e5b9e7f2b1d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:50.935163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:50.935857 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:50.936214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:50.936414 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] PATH_INFO: `/v3/projects/90bf68a46ae0410faaee4503b7b51cce/users/0ef2dcae16464ebfa18ff0f803d569ea/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:50.956853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bf68a46ae0410faaee4503b7b51cce, user_id=0ef2dcae16464ebfa18ff0f803d569ea, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:50.959724 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:51.078892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1329/2637] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:50 2022] PUT /identity/v3/projects/90bf68a46ae0410faaee4503b7b51cce/users/0ef2dcae16464ebfa18ff0f803d569ea/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:51.084112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:51.091359 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:51.095012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4d03-2797-439c-b5b0-049fc7a46014 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:51.162158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea285c-7f47-4668-9046-6438b2118846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:51.162801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea285c-7f47-4668-9046-6438b2118846 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:51.163115 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea285c-7f47-4668-9046-6438b2118846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:51.163397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea285c-7f47-4668-9046-6438b2118846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:51.164734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea285c-7f47-4668-9046-6438b2118846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:51.166463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea285c-7f47-4668-9046-6438b2118846 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:51.177216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1309/2638] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:51 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:51.182425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:51.192954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3548b-640d-42e7-b455-a52687f66429 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:51.198433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:51.198673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:51.198931 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:51.199246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] PATH_INFO: `/v3/projects/90bf68a46ae0410faaee4503b7b51cce/users/0ef2dcae16464ebfa18ff0f803d569ea/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:51.201608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bf68a46ae0410faaee4503b7b51cce, user_id=0ef2dcae16464ebfa18ff0f803d569ea, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:51.204328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:51.225622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1330/2639] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:51 2022] PUT /identity/v3/projects/90bf68a46ae0410faaee4503b7b51cce/users/0ef2dcae16464ebfa18ff0f803d569ea/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:51.236664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2c8ef-5152-4d93-a48f-3aba5200aee4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:51.237222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2c8ef-5152-4d93-a48f-3aba5200aee4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:51.237222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2c8ef-5152-4d93-a48f-3aba5200aee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:51.278197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-97f2c8ef-5152-4d93-a48f-3aba5200aee4 None None] MFA Rules not processed for user `0ef2dcae16464ebfa18ff0f803d569ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:51.278197 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97f2c8ef-5152-4d93-a48f-3aba5200aee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:51.361387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1310/2640] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:51.373381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:51.375166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:51.447408 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef57bb-ebc9-465f-81c7-8315d878f501 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:51.468778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:51.469975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:51.469975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:51.470223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:51.481632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:51.483497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:51.495066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1331/2641] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:51.531391 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2c8ef-5152-4d93-a48f-3aba5200aee4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:51.622007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2c8ef-5152-4d93-a48f-3aba5200aee4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:51.627144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:51.627808 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:51.628092 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:51.628373 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] PATH_INFO: `/v3/users/a9258de001484e7b9d512f3d79dced2d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:51.629658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9258de001484e7b9d512f3d79dced2d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:51.631690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:51.641953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:51.653652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50bb24-b938-43e0-ba4f-02e1a4704b53 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:51.658567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:51.659235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:51.659499 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:51.659775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] PATH_INFO: `/v3/users/275ec79bbf594b9b8c9245d37c38ed64` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:51.661182 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=275ec79bbf594b9b8c9245d37c38ed64)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:51.663114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:51.765634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:51.846690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9258de001484e7b9d512f3d79dced2d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.853711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9258de001484e7b9d512f3d79dced2d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.854411 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9258de001484e7b9d512f3d79dced2d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.874751 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275ec79bbf594b9b8c9245d37c38ed64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.875520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275ec79bbf594b9b8c9245d37c38ed64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.884292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275ec79bbf594b9b8c9245d37c38ed64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.897420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275ec79bbf594b9b8c9245d37c38ed64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.923751 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9258de001484e7b9d512f3d79dced2d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.924484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-087d92e7-9fd8-4974-9c7a-58b8f1627022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9258de001484e7b9d512f3d79dced2d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:51.944867 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1311/2642] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:51 2022] DELETE /identity/v3/users/a9258de001484e7b9d512f3d79dced2d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:51.951097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:51.951387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:51.951672 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:51.971590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:51.972851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:52.026171 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cc05880e-f07b-433b-afc2-da6eac422ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275ec79bbf594b9b8c9245d37c38ed64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:52.032983 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1332/2643] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:51 2022] DELETE /identity/v3/users/275ec79bbf594b9b8c9245d37c38ed64 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:52.037359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92021f9e-c6b7-48a1-a8ed-4e93e5c97373 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:52.037650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92021f9e-c6b7-48a1-a8ed-4e93e5c97373 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:52.037928 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92021f9e-c6b7-48a1-a8ed-4e93e5c97373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:52.042151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1312/2644] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:52.052944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:52.054389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:52.059687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-92021f9e-c6b7-48a1-a8ed-4e93e5c97373 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:52.061093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-92021f9e-c6b7-48a1-a8ed-4e93e5c97373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:52.104272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:52.114492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788e1eb-08dc-469c-b60f-b300258a2027 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:52.138954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1333/2645] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:52.149863 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92021f9e-c6b7-48a1-a8ed-4e93e5c97373 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:52.170367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:52.171264 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:52.171552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:52.171836 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:52.182288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:52.183759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:52.193922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1313/2646] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:52.209194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92021f9e-c6b7-48a1-a8ed-4e93e5c97373 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:52.214327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:52.214967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:52.215247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:52.215524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:52.225960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:52.228265 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:52.239731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1334/2647] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:52.880487 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:52.882085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:52.894611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:52.898795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91a3a-fd52-4aee-82ce-0110844c96fe service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:52.904186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:52.904591 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:52.905445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:52.905729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] PATH_INFO: `/v3/projects/444cb09d8b3f4d2db96274f984b31c97` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:52.907158 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444cb09d8b3f4d2db96274f984b31c97)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:52.909408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:52.925150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:52.928708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:52.951142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e813b-a3e8-4552-ac08-e4b65abe8ebc service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:52.958099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:52.958845 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:52.959564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:52.959847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] PATH_INFO: `/v3/projects/c707fd49a10e4806a8e33440be5c9f37` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:52.961784 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c707fd49a10e4806a8e33440be5c9f37)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:52.963802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:53.064255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444cb09d8b3f4d2db96274f984b31c97', 'request_id': 'req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.082852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c707fd49a10e4806a8e33440be5c9f37', 'request_id': 'req-e7cfafef-2422-433e-a405-9591d8406c6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.090115 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c707fd49a10e4806a8e33440be5c9f37', 'request_id': 'req-e7cfafef-2422-433e-a405-9591d8406c6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.097181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c707fd49a10e4806a8e33440be5c9f37', 'request_id': 'req-e7cfafef-2422-433e-a405-9591d8406c6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.123389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444cb09d8b3f4d2db96274f984b31c97', 'request_id': 'req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.130126 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444cb09d8b3f4d2db96274f984b31c97', 'request_id': 'req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.145335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] The token cache is being invalidate because project 444cb09d8b3f4d2db96274f984b31c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:53.146894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.146894 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.148750 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1314/2648] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:52 2022] DELETE /identity/v3/projects/444cb09d8b3f4d2db96274f984b31c97 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:53.152844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:53.154329 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:53.163347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] The token cache is being invalidate because project c707fd49a10e4806a8e33440be5c9f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:53.164136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.165099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.166637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1335/2649] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:52 2022] DELETE /identity/v3/projects/c707fd49a10e4806a8e33440be5c9f37 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:53.170560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:53.172655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:53.230734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a94b6-2d99-4839-a9a4-5b822416c3d9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:53.254373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfafef-2422-433e-a405-9591d8406c6e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:53.261395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:53.262201 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:53.262594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:53.262979 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] PATH_INFO: `/v3/users/c90f4a8449314c9fada85f8a050f03d2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:53.264891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90f4a8449314c9fada85f8a050f03d2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:53.269078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:53.276655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:53.305444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:53.306099 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:53.306390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:53.306666 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] PATH_INFO: `/v3/users/0ef2dcae16464ebfa18ff0f803d569ea` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:53.307890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef2dcae16464ebfa18ff0f803d569ea)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:53.310073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:53.505341 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90f4a8449314c9fada85f8a050f03d2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.505993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90f4a8449314c9fada85f8a050f03d2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.514053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90f4a8449314c9fada85f8a050f03d2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.528360 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90f4a8449314c9fada85f8a050f03d2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.544677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef2dcae16464ebfa18ff0f803d569ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.552841 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef2dcae16464ebfa18ff0f803d569ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.553553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef2dcae16464ebfa18ff0f803d569ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.633493 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90f4a8449314c9fada85f8a050f03d2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.642591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1336/2650] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:53 2022] DELETE /identity/v3/users/c90f4a8449314c9fada85f8a050f03d2 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:53.692716 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef2dcae16464ebfa18ff0f803d569ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.693417 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef2dcae16464ebfa18ff0f803d569ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:53.719045 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1315/2651] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:53 2022] DELETE /identity/v3/users/0ef2dcae16464ebfa18ff0f803d569ea => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:53.792788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:54.262742 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:54.264239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:54.305072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:54.309255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:54.342102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fefd69-d22f-49e4-9494-072d9753e4da admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:54.384650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345658da-bbe3-4c14-8d4b-414047c9a055 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:54.393364 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:54.394027 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.394315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.394596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] PATH_INFO: `/v3/projects/90bf68a46ae0410faaee4503b7b51cce` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.395781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90bf68a46ae0410faaee4503b7b51cce)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:54.397761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:54.412016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:54.412708 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.412995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.413272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] PATH_INFO: `/v3/projects/555781c028514feb9a76a31fa0b3bbab` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.414649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=555781c028514feb9a76a31fa0b3bbab)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:54.416602 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:54.495736 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90bf68a46ae0410faaee4503b7b51cce', 'request_id': 'req-6c930acf-0ee2-40da-98f9-62089b362b64'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.529824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:54.542144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555781c028514feb9a76a31fa0b3bbab', 'request_id': 'req-9372f2bd-0faa-4f13-97ac-896064dca776'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.548787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555781c028514feb9a76a31fa0b3bbab', 'request_id': 'req-9372f2bd-0faa-4f13-97ac-896064dca776'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.555043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555781c028514feb9a76a31fa0b3bbab', 'request_id': 'req-9372f2bd-0faa-4f13-97ac-896064dca776'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.586669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90bf68a46ae0410faaee4503b7b51cce', 'request_id': 'req-6c930acf-0ee2-40da-98f9-62089b362b64'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.592916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90bf68a46ae0410faaee4503b7b51cce', 'request_id': 'req-6c930acf-0ee2-40da-98f9-62089b362b64'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.604885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] The token cache is being invalidate because project 90bf68a46ae0410faaee4503b7b51cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:54.605178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.606333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6c930acf-0ee2-40da-98f9-62089b362b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.607912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1316/2652] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:54 2022] DELETE /identity/v3/projects/90bf68a46ae0410faaee4503b7b51cce => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:54.612653 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b6ae-d28b-43ed-bf7e-e8beea119b57 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.612938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b6ae-d28b-43ed-bf7e-e8beea119b57 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.613295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b6ae-d28b-43ed-bf7e-e8beea119b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.633731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bba9b6ae-d28b-43ed-bf7e-e8beea119b57 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:54.634962 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bba9b6ae-d28b-43ed-bf7e-e8beea119b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:54.672436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] The token cache is being invalidate because project 555781c028514feb9a76a31fa0b3bbab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:54.672743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.673396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9372f2bd-0faa-4f13-97ac-896064dca776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:54.675112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1337/2653] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:54 2022] DELETE /identity/v3/projects/555781c028514feb9a76a31fa0b3bbab => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:54.680488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e299ad-1f8a-4b73-9c95-fd4d6c90549f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.680792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e299ad-1f8a-4b73-9c95-fd4d6c90549f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.681070 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e299ad-1f8a-4b73-9c95-fd4d6c90549f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.690523 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:54.697147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1317/2654] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:54.701823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-39e299ad-1f8a-4b73-9c95-fd4d6c90549f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:54.702319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39e299ad-1f8a-4b73-9c95-fd4d6c90549f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:54.709345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aeccd-98ef-4cf4-bba8-78360f9866d6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.709633 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aeccd-98ef-4cf4-bba8-78360f9866d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.709909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aeccd-98ef-4cf4-bba8-78360f9866d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.710411 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1338/2655] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:54.720194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b32ec-bf3d-428f-9682-2d2c93285f16 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.720506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b32ec-bf3d-428f-9682-2d2c93285f16 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.720781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b32ec-bf3d-428f-9682-2d2c93285f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.730379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-574aeccd-98ef-4cf4-bba8-78360f9866d6 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:54.731964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-574aeccd-98ef-4cf4-bba8-78360f9866d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:54.740137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1318/2656] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:54.741658 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-913b32ec-bf3d-428f-9682-2d2c93285f16 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:54.747629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-913b32ec-bf3d-428f-9682-2d2c93285f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:54.750620 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aeccd-98ef-4cf4-bba8-78360f9866d6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:54.751951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1339/2657] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:54.755942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b32ec-bf3d-428f-9682-2d2c93285f16 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:54.760855 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aeccd-98ef-4cf4-bba8-78360f9866d6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:54.765220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b32ec-bf3d-428f-9682-2d2c93285f16 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:54.765941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:54.766689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.767074 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.767447 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.768220 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:54.768571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:54.770044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:54.770236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:54.770641 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.770922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.771198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.771889 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:43:54.772193 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:54.773981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:54.783218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1319/2658] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:43:54.785730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1340/2659] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:43:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:43:54.787382 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:54.790517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:54.799294 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dfe74-f740-441e-af74-96a132bb4745 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:54.800241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead929a-931f-4b0a-aeed-7156559579de admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:54.803794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:54.805378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:54.805700 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.805972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:54.806077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.806250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.806349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:54.806529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:54.806929 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:54.807183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:54.809008 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:54.809307 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:54.931872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1320/2660] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/projects => generated 366 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:54.935934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:54.951209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1341/2661] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/projects => generated 380 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:54.952351 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.012296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d255e-54d5-4773-a68e-09c44ee489f1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.017330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac244d4-f2df-4d88-8d1b-012ae4be85a3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.017552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.018253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.018547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.018827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.019512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.021512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.022662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.024001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.024354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.024730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.025508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.027497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.140243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1342/2662] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:43:55.144623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.144909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1321/2663] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:54 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:55.146247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.155874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07242650-8490-4cbf-bc75-663c073cbe37 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.160952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.161575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.161897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.162179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.162891 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.165825 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.169669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557f63f-1980-4321-8880-47c2a9b8bd4b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.180693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1343/2664] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:55 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:55.180907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.181553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.181874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.182163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.182895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.184539 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.185715 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.196577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1322/2665] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:55 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:55.200882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.205401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa790-43a6-42d6-8c8d-5b11440e22a1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.212745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.213592 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.213795 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.214644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.214774 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e9d4e-8bbb-4734-949c-4b11636e6aa8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.214894 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.217123 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.220636 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.221399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.221788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.222437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.223313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.225153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.240546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1323/2666] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:55 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:55.244356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.244958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1344/2667] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:55 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:55.249836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.259674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445cc7a-760c-4d62-8cff-0a6a0159d782 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.264136 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966de92d-0898-47b6-9e61-a6ec5479a50e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.271381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7086d-6503-4f60-b18b-d6c3d2777f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.271498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c871e-84f6-4111-8fdf-2f2d909d9411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.271674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7086d-6503-4f60-b18b-d6c3d2777f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.271674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7086d-6503-4f60-b18b-d6c3d2777f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.271883 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c871e-84f6-4111-8fdf-2f2d909d9411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.271975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7086d-6503-4f60-b18b-d6c3d2777f9c admin admin] PATH_INFO: `/v3/projects/c738705f93014d42963be64cac885ed7/users/d94860048aab427983694d3ccdd29029/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.272067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c871e-84f6-4111-8fdf-2f2d909d9411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.272067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c871e-84f6-4111-8fdf-2f2d909d9411 admin admin] PATH_INFO: `/v3/projects/67bed7dd7b414caeb01b965cb434a681/users/d3ddd4abfdbd4c4ca2220d2719700f52/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.289870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7086d-6503-4f60-b18b-d6c3d2777f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c738705f93014d42963be64cac885ed7, user_id=d94860048aab427983694d3ccdd29029, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.292783 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7086d-6503-4f60-b18b-d6c3d2777f9c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.302726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c871e-84f6-4111-8fdf-2f2d909d9411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bed7dd7b414caeb01b965cb434a681, user_id=d3ddd4abfdbd4c4ca2220d2719700f52, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.306010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c871e-84f6-4111-8fdf-2f2d909d9411 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.389730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1324/2668] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:55 2022] PUT /identity/v3/projects/67bed7dd7b414caeb01b965cb434a681/users/d3ddd4abfdbd4c4ca2220d2719700f52/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:55.391312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1345/2669] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:55 2022] PUT /identity/v3/projects/c738705f93014d42963be64cac885ed7/users/d94860048aab427983694d3ccdd29029/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:55.402908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b25b-205a-4c56-9a76-8f98e5bf3b56 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.403471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b25b-205a-4c56-9a76-8f98e5bf3b56 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.404454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b25b-205a-4c56-9a76-8f98e5bf3b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.437903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-44b9b25b-205a-4c56-9a76-8f98e5bf3b56 None None] MFA Rules not processed for user `d94860048aab427983694d3ccdd29029`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:55.439162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-44b9b25b-205a-4c56-9a76-8f98e5bf3b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:55.463476 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070b41-3d3c-45ad-b63b-a9dc742f0032 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.464046 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070b41-3d3c-45ad-b63b-a9dc742f0032 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.464570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070b41-3d3c-45ad-b63b-a9dc742f0032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.493591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e1070b41-3d3c-45ad-b63b-a9dc742f0032 None None] MFA Rules not processed for user `d3ddd4abfdbd4c4ca2220d2719700f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:55.495361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e1070b41-3d3c-45ad-b63b-a9dc742f0032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:55.500991 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:55.511206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1325/2670] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:55.516075 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9b25b-205a-4c56-9a76-8f98e5bf3b56 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.572963 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1346/2671] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:43:55.577506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1070b41-3d3c-45ad-b63b-a9dc742f0032 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.584885 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9b25b-205a-4c56-9a76-8f98e5bf3b56 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.589474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1070b41-3d3c-45ad-b63b-a9dc742f0032 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.590250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.590895 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.591183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.591457 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.592139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.594175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.594392 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.595024 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.595280 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.595577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.596235 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.598192 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.668704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:55.676194 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1326/2672] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:55 2022] POST /identity/v3/projects => generated 382 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:55.680190 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.731781 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1347/2673] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:43:55 2022] POST /identity/v3/projects => generated 366 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:55.735829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.751971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c41b44-c3c5-4019-9370-3913ff8cc4ac admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.805682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c7377-0487-454e-af94-ecb0074b4cac admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.810540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.811169 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.811452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.811729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.812399 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.812890 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.813525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.813807 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.814084 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.814507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.814905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.816839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.941959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1348/2674] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:55 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:55.942458 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1327/2675] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:43:55 2022] POST /identity/v3/users => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:43:55.946194 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.946334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.956321 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec9e0f-c8e7-468b-9a91-4e0c94f9dccf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.956635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38468dce-1493-4a4e-84df-a7bd7feee0d6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:55.961673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.961852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:55.962311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.962431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:55.962596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.962698 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:55.962874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.962982 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:55.964637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.964826 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:55.965230 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.965349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:55.976102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1328/2676] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:55 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:55.980187 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1349/2677] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:55 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:55.989845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:55.990150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.000681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400691-90be-4d3e-b1f9-d7c2109dbddd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.000886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8aae-5acf-4f87-bf0c-438750a94fe9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.005495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.006175 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.006306 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.006491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.006769 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] PATH_INFO: `/v3/projects/d733c82562104893a1c14ab50f3b1865/users/e9cae1ab68f54e1a9af3af93d29413f9/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.006901 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.007110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.007374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] PATH_INFO: `/v3/projects/a78ae28423f34950bebd16389187df4b/users/cf158dc2607742939fe81aabf49e14db/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.025297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d733c82562104893a1c14ab50f3b1865, user_id=e9cae1ab68f54e1a9af3af93d29413f9, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.028037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.029690 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78ae28423f34950bebd16389187df4b, user_id=cf158dc2607742939fe81aabf49e14db, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.032396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.113003 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1329/2679] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:55 2022] PUT /identity/v3/projects/a78ae28423f34950bebd16389187df4b/users/cf158dc2607742939fe81aabf49e14db/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:56.113257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1350/2679] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:55 2022] PUT /identity/v3/projects/d733c82562104893a1c14ab50f3b1865/users/e9cae1ab68f54e1a9af3af93d29413f9/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:56.117017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.117221 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.181763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c52974-2588-4f8f-b0da-ccd3228e7562 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.182643 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73e11-bde3-4682-966d-8d4f843f208b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.184136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:56.187617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.193495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.193625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.193728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.193728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.193903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.193903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.193903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.194141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.194221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.194298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.194384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.202290 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1330/2680] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:56 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:56.202588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1351/2681] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:43:56 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:43:56.206304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.206453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.214251 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:56.217525 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84feae4-bd53-40ae-8a03-de106dffe647 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.218013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90aeef-6faf-4364-b684-a56c1fe8e789 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.222286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177550d1-3545-4ff7-9a1b-47aeb5147a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.222672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14a7f6-c990-4ac4-b557-6bda83001986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.222960 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177550d1-3545-4ff7-9a1b-47aeb5147a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.223251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177550d1-3545-4ff7-9a1b-47aeb5147a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.223368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14a7f6-c990-4ac4-b557-6bda83001986 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.223559 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177550d1-3545-4ff7-9a1b-47aeb5147a79 admin admin] PATH_INFO: `/v3/projects/d733c82562104893a1c14ab50f3b1865/users/e9cae1ab68f54e1a9af3af93d29413f9/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.223673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14a7f6-c990-4ac4-b557-6bda83001986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.223858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14a7f6-c990-4ac4-b557-6bda83001986 admin admin] PATH_INFO: `/v3/projects/a78ae28423f34950bebd16389187df4b/users/cf158dc2607742939fe81aabf49e14db/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.225568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177550d1-3545-4ff7-9a1b-47aeb5147a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d733c82562104893a1c14ab50f3b1865, user_id=e9cae1ab68f54e1a9af3af93d29413f9, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.225980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14a7f6-c990-4ac4-b557-6bda83001986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78ae28423f34950bebd16389187df4b, user_id=cf158dc2607742939fe81aabf49e14db, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.229127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177550d1-3545-4ff7-9a1b-47aeb5147a79 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.229599 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14a7f6-c990-4ac4-b557-6bda83001986 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.239812 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1331/2682] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:56 2022] PUT /identity/v3/projects/d733c82562104893a1c14ab50f3b1865/users/e9cae1ab68f54e1a9af3af93d29413f9/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:43:56.240410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1352/2683] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:43:56 2022] PUT /identity/v3/projects/a78ae28423f34950bebd16389187df4b/users/cf158dc2607742939fe81aabf49e14db/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:56.250974 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.251175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.251278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.251424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.251607 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.251748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.282256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] MFA Rules not processed for user `e9cae1ab68f54e1a9af3af93d29413f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:56.283534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:56.283776 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] MFA Rules not processed for user `cf158dc2607742939fe81aabf49e14db`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:56.285272 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:56.364365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1353/2684] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:56.370122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1332/2685] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:56.376577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.381170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:56.382078 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.383887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:56.450922 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d578643-d510-4ec7-a8db-8f6b6a0a7b2d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.452965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5478b81-2294-451a-a249-4d42d7ecf983 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.461210 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:56.464567 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.465044 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.465311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.465603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.465886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.466003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.466290 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.466635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.475596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.476122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.477829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.478143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.487734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1354/2687] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:56.487963 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1333/2687] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:43:56.783759 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:56.793795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a3b0c-ac44-45ed-946a-70b4cabace96 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:56.798063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9ac7e-c2b3-44c4-a626-47bbaf138e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:56.798735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ac7e-c2b3-44c4-a626-47bbaf138e82 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:56.799020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ac7e-c2b3-44c4-a626-47bbaf138e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:56.799309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ac7e-c2b3-44c4-a626-47bbaf138e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:56.867735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9ac7e-c2b3-44c4-a626-47bbaf138e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:56.868893 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9ac7e-c2b3-44c4-a626-47bbaf138e82 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:56.878503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1334/2688] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:57.096558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:57.151061 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:57.166146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9bb4e-2e2f-4fd0-8431-4c42f2d8a787 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:57.171190 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:57.171841 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:57.172124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:57.172402 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] PATH_INFO: `/v3/users/d94860048aab427983694d3ccdd29029` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:57.173812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d94860048aab427983694d3ccdd29029)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:57.178427 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:57.372903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd94860048aab427983694d3ccdd29029'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:57.373644 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd94860048aab427983694d3ccdd29029'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:57.380786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd94860048aab427983694d3ccdd29029'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:57.393875 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd94860048aab427983694d3ccdd29029'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:57.471822 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd94860048aab427983694d3ccdd29029'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:57.483572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1355/2689] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:57 2022] DELETE /identity/v3/users/d94860048aab427983694d3ccdd29029 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:57.484283 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:57.484853 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:57.484954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:57.504906 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:57.506097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:57.574387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1335/2690] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:43:57.665483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:57.666820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:57.732241 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a78b8-9325-4c6f-98f3-09c8bf3f4422 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:57.736016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:57.736672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:57.737619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:57.737902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:57.748375 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:57.749866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:57.760418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1356/2691] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:57.909101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:57.910514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:57.923915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a1bfb-047f-42d9-a876-3797e8417140 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:57.928626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:57.928626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:57.928626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:57.929838 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:57.931449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:58.000137 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:58.001650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:58.012764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1336/2692] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:43:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:43:58.297929 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:58.299474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:58.313765 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eca89f-d1d3-47ce-a730-9d862ec4a04f service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:58.319032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:58.320359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:58.320359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:58.321260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] PATH_INFO: `/v3/projects/c738705f93014d42963be64cac885ed7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:58.322548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c738705f93014d42963be64cac885ed7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:58.324517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:58.443349 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c738705f93014d42963be64cac885ed7', 'request_id': 'req-e41ed605-dc29-4c0b-a369-449b732b1874'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.455470 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c738705f93014d42963be64cac885ed7', 'request_id': 'req-e41ed605-dc29-4c0b-a369-449b732b1874'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.468396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c738705f93014d42963be64cac885ed7', 'request_id': 'req-e41ed605-dc29-4c0b-a369-449b732b1874'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.470972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:58.574828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] The token cache is being invalidate because project c738705f93014d42963be64cac885ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:58.575054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.575670 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.577239 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1357/2693] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:58 2022] DELETE /identity/v3/projects/c738705f93014d42963be64cac885ed7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:58.581506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:58.593012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407344b-d533-4474-ab3a-0696c934d9a8 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:58.596693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:43:58.655503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:58.656179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:58.656437 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:58.656743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] PATH_INFO: `/v3/users/cf158dc2607742939fe81aabf49e14db` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:58.658001 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf158dc2607742939fe81aabf49e14db)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:58.660056 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:58.893209 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf158dc2607742939fe81aabf49e14db'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.901478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf158dc2607742939fe81aabf49e14db'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.902202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf158dc2607742939fe81aabf49e14db'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.986534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf158dc2607742939fe81aabf49e14db'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:58.987150 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d09a93e-0b01-4fc9-a70f-5ee5dcec8865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf158dc2607742939fe81aabf49e14db'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:59.006709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1337/2694] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:43:58 2022] DELETE /identity/v3/users/cf158dc2607742939fe81aabf49e14db => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:59.577275 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:43:59.578492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:59.610056 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ed605-dc29-4c0b-a369-449b732b1874 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:43:59.674195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:43:59.674874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:59.675152 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:59.675428 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] PATH_INFO: `/v3/projects/a78ae28423f34950bebd16389187df4b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:59.676605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a78ae28423f34950bebd16389187df4b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:43:59.678569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:43:59.806386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78ae28423f34950bebd16389187df4b', 'request_id': 'req-1363c531-4b49-4039-9c76-0da046691f00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:59.814165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78ae28423f34950bebd16389187df4b', 'request_id': 'req-1363c531-4b49-4039-9c76-0da046691f00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:59.823196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78ae28423f34950bebd16389187df4b', 'request_id': 'req-1363c531-4b49-4039-9c76-0da046691f00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:59.895058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] The token cache is being invalidate because project a78ae28423f34950bebd16389187df4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:43:59.895345 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:59.896008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1363c531-4b49-4039-9c76-0da046691f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:43:59.897635 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1358/2695] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:43:59 2022] DELETE /identity/v3/projects/a78ae28423f34950bebd16389187df4b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:43:59.905393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2b10d-190e-4775-baef-6b9d333e7159 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:43:59.905393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2b10d-190e-4775-baef-6b9d333e7159 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:43:59.905949 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2b10d-190e-4775-baef-6b9d333e7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:43:59.928022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0bb2b10d-190e-4775-baef-6b9d333e7159 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:43:59.929489 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0bb2b10d-190e-4775-baef-6b9d333e7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:43:59.993231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1338/2696] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:00.000833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3a8c3-69a4-4b68-a258-a46bff0ddcbb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.001133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3a8c3-69a4-4b68-a258-a46bff0ddcbb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.001435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3a8c3-69a4-4b68-a258-a46bff0ddcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.022304 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-95e3a8c3-69a4-4b68-a258-a46bff0ddcbb None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:00.024215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-95e3a8c3-69a4-4b68-a258-a46bff0ddcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:00.032739 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1359/2697] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:00.037036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2b10d-190e-4775-baef-6b9d333e7159 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.050845 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2b10d-190e-4775-baef-6b9d333e7159 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.052938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.053573 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.053861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.054131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.055826 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:44:00.056151 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.058017 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:00.074327 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1339/2698] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:44:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:44:00.075775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3a8c3-69a4-4b68-a258-a46bff0ddcbb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.086344 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3a8c3-69a4-4b68-a258-a46bff0ddcbb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.091191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.091827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.092113 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.092420 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.093179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.095114 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:00.184434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1360/2699] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:00 2022] POST /identity/v3/projects => generated 372 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:00.191136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.202766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129a62-c632-4590-bbdd-1508638e4d27 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.214130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:00.273589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.274580 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.275043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.275449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.276140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.278335 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:00.352401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1340/2700] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:00 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:00.356862 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.369410 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542e39a-950d-4a62-b665-7eac82c2cf90 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.374778 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.375540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.375834 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.376110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.376856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.378464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:00.390590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1361/2701] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:00 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:00.394774 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.406971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea4732-e471-41ff-bf5f-fe0261c52a7a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.414891 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86ede5-c414-4c2a-ae28-84e960676c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.415537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86ede5-c414-4c2a-ae28-84e960676c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.415992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86ede5-c414-4c2a-ae28-84e960676c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.416225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86ede5-c414-4c2a-ae28-84e960676c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.417703 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86ede5-c414-4c2a-ae28-84e960676c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.419408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86ede5-c414-4c2a-ae28-84e960676c54 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:00.433177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1341/2702] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:00 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:00.437371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.448693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f659e57-d4ef-4b6d-bd4d-28daeddfd167 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.460298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.462103 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.462595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.465212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] PATH_INFO: `/v3/projects/3e2d0e1f983f40cc8635382e76923fd9/users/378033d7851e42329dfd9b94c599cfe0/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.483264 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2d0e1f983f40cc8635382e76923fd9, user_id=378033d7851e42329dfd9b94c599cfe0, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.486144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:00.614129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1362/2703] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:00 2022] PUT /identity/v3/projects/3e2d0e1f983f40cc8635382e76923fd9/users/378033d7851e42329dfd9b94c599cfe0/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:00.625599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5298-673e-413a-ab4d-5c53e8a28193 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.627395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5298-673e-413a-ab4d-5c53e8a28193 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.627395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5298-673e-413a-ab4d-5c53e8a28193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.661340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-10bb5298-673e-413a-ab4d-5c53e8a28193 None None] MFA Rules not processed for user `378033d7851e42329dfd9b94c599cfe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:00.662712 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-10bb5298-673e-413a-ab4d-5c53e8a28193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:00.745613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1342/2704] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:00.768233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.776352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:00.806393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:00.845090 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1ed6-3f17-4bd8-8756-c9b4bb54b6cd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.848911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f85d5-8aae-4227-9935-8ff97dbf2e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.849530 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f85d5-8aae-4227-9935-8ff97dbf2e3e service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.849812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f85d5-8aae-4227-9935-8ff97dbf2e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.850170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f85d5-8aae-4227-9935-8ff97dbf2e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.866853 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f85d5-8aae-4227-9935-8ff97dbf2e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.868330 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f85d5-8aae-4227-9935-8ff97dbf2e3e service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:00.878570 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1363/2705] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:00.906186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5298-673e-413a-ab4d-5c53e8a28193 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:00.974315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5298-673e-413a-ab4d-5c53e8a28193 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:00.983435 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:00.984133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:00.986168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:00.986168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] PATH_INFO: `/v3/users/378033d7851e42329dfd9b94c599cfe0` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:00.986168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=378033d7851e42329dfd9b94c599cfe0)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:00.988012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:01.013057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:01.168401 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '378033d7851e42329dfd9b94c599cfe0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:01.176206 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '378033d7851e42329dfd9b94c599cfe0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:01.176966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '378033d7851e42329dfd9b94c599cfe0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:01.271390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '378033d7851e42329dfd9b94c599cfe0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:01.272016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '378033d7851e42329dfd9b94c599cfe0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:01.290852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1343/2706] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:00 2022] DELETE /identity/v3/users/378033d7851e42329dfd9b94c599cfe0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:01.295707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:01.296097 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:01.296386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:01.315729 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:01.321478 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:01.391442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1364/2707] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:01.404108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:01.405783 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:01.468706 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cfa31-391b-41f9-86f0-aa2dfcfa0a45 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:01.472538 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec034e3-2b1e-4ddd-ac39-e906a4f5bf5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:01.473183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec034e3-2b1e-4ddd-ac39-e906a4f5bf5b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:01.473503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec034e3-2b1e-4ddd-ac39-e906a4f5bf5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:01.473781 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec034e3-2b1e-4ddd-ac39-e906a4f5bf5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:01.483848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec034e3-2b1e-4ddd-ac39-e906a4f5bf5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:01.485315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec034e3-2b1e-4ddd-ac39-e906a4f5bf5b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:01.495450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1344/2708] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:01.955872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:01.957388 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:01.973545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9d8db-621e-4f87-8008-4f854fbeaa3b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:01.975474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:01.976448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:01.976829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:01.977119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] PATH_INFO: `/v3/projects/3e2d0e1f983f40cc8635382e76923fd9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:01.978329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2d0e1f983f40cc8635382e76923fd9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:01.980279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:02.168685 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2d0e1f983f40cc8635382e76923fd9', 'request_id': 'req-8a8b8981-7da4-4702-a89e-7434db23248c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:02.175431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2d0e1f983f40cc8635382e76923fd9', 'request_id': 'req-8a8b8981-7da4-4702-a89e-7434db23248c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:02.182881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2d0e1f983f40cc8635382e76923fd9', 'request_id': 'req-8a8b8981-7da4-4702-a89e-7434db23248c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:02.326352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] The token cache is being invalidate because project 3e2d0e1f983f40cc8635382e76923fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:02.326352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:02.327141 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a8b8981-7da4-4702-a89e-7434db23248c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:02.329131 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1365/2709] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:01 2022] DELETE /identity/v3/projects/3e2d0e1f983f40cc8635382e76923fd9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:02.334174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453aa61-c7f1-41bf-a7dd-2daad862775a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.334631 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453aa61-c7f1-41bf-a7dd-2daad862775a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.334729 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453aa61-c7f1-41bf-a7dd-2daad862775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.356186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5453aa61-c7f1-41bf-a7dd-2daad862775a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:02.358046 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5453aa61-c7f1-41bf-a7dd-2daad862775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:02.423188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1345/2710] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:02.432692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7cd10-026c-4494-a7ef-bf05b54236d1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.433002 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7cd10-026c-4494-a7ef-bf05b54236d1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.433288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7cd10-026c-4494-a7ef-bf05b54236d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.453952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-38d7cd10-026c-4494-a7ef-bf05b54236d1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:02.455424 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-38d7cd10-026c-4494-a7ef-bf05b54236d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:02.464836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1366/2711] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:02.469958 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453aa61-c7f1-41bf-a7dd-2daad862775a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:02.480947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453aa61-c7f1-41bf-a7dd-2daad862775a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:02.485908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:02.486548 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.486833 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.487130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.487965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:44:02.488278 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:02.489970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:02.502638 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1346/2712] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:44:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:44:02.507147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7cd10-026c-4494-a7ef-bf05b54236d1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:02.516507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7cd10-026c-4494-a7ef-bf05b54236d1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:02.521612 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:02.521842 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.522120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.522390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.523682 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:02.525632 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:02.667958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1367/2713] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:02 2022] POST /identity/v3/projects => generated 376 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:02.672560 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:02.683251 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e17788-249f-42bc-8053-27fe00af2ffa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:02.713208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:02.740149 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:02.740810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.741095 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.741379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.742600 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:02.744618 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:02.859973 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1347/2714] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:02 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:02.864283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:02.875743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffba4e-82a6-412a-ac5e-bac6491c1ccc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:02.880412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:02.881128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.881412 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.884256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.884256 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:02.884452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:02.895180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1368/2715] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:02 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:02.899879 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:02.910739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9c9f5-58d4-4229-b8ab-d6e9503841d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:02.917242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1bdb9-dded-4012-b892-088a8fb1e0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:02.917872 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1bdb9-dded-4012-b892-088a8fb1e0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.918188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1bdb9-dded-4012-b892-088a8fb1e0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.918470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1bdb9-dded-4012-b892-088a8fb1e0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.919178 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1bdb9-dded-4012-b892-088a8fb1e0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:02.921125 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1bdb9-dded-4012-b892-088a8fb1e0c6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:02.933374 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1348/2716] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:02 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:02.937879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:02.947870 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096cce3-8b84-4aa6-b1ba-458f216246f2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:02.952536 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:02.953230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:02.953518 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:02.953803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] PATH_INFO: `/v3/projects/5b03a9ff5cb848ce9c77c39120dc80bb/users/0f5972cbf5a245d583e59a09d509320c/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:02.956110 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:02.969413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b03a9ff5cb848ce9c77c39120dc80bb, user_id=0f5972cbf5a245d583e59a09d509320c, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:02.972378 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:03.129528 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1369/2717] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:02 2022] PUT /identity/v3/projects/5b03a9ff5cb848ce9c77c39120dc80bb/users/0f5972cbf5a245d583e59a09d509320c/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:03.142860 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa7ed-fa5f-4b41-a6ff-67006dce7dec None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:03.143314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa7ed-fa5f-4b41-a6ff-67006dce7dec None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:03.143599 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa7ed-fa5f-4b41-a6ff-67006dce7dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:03.175778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-05afa7ed-fa5f-4b41-a6ff-67006dce7dec None None] MFA Rules not processed for user `0f5972cbf5a245d583e59a09d509320c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:03.176991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-05afa7ed-fa5f-4b41-a6ff-67006dce7dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:03.245066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1349/2718] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:03.249831 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:03.312704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f471e-9ee5-45fb-bd23-1270f9aee620 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:03.317492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:03.318109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:03.318395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:03.318674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:03.319381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:03.322019 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:03.409701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1370/2719] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:03 2022] POST /identity/v3/projects => generated 378 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:03.413709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afa7ed-fa5f-4b41-a6ff-67006dce7dec None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:03.424612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afa7ed-fa5f-4b41-a6ff-67006dce7dec None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:03.483035 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:03.483656 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:03.483941 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:03.484249 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:03.484982 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:03.486961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:03.569096 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1350/2720] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:03 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:03.574163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:03.585100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cbe5e-78bd-4f2f-ba00-6dab64596cdd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:03.590615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:03.591324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:03.591662 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:03.597346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:03.597346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:03.597346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:03.607434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1371/2721] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:03 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:03.611701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:03.622161 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff717a75-2985-434a-b982-143f4706b2e3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:03.627223 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341306-f37a-4c8a-8612-4818888e4d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:03.627852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341306-f37a-4c8a-8612-4818888e4d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:03.628138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341306-f37a-4c8a-8612-4818888e4d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:03.628413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341306-f37a-4c8a-8612-4818888e4d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:03.629145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341306-f37a-4c8a-8612-4818888e4d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:03.630740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341306-f37a-4c8a-8612-4818888e4d2e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:03.642801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1351/2722] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:03 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:03.647173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:03.667513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cce3d-069f-4f1e-9111-16b29ac39c77 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:03.673035 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:03.673706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:03.673989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:03.674269 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] PATH_INFO: `/v3/projects/8c06a06e8f34435995766e8973a44262/users/f7f9b61b573243089c5b68348fc57ee5/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:03.690847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c06a06e8f34435995766e8973a44262, user_id=f7f9b61b573243089c5b68348fc57ee5, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:03.693554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:03.797832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1372/2723] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:03 2022] PUT /identity/v3/projects/8c06a06e8f34435995766e8973a44262/users/f7f9b61b573243089c5b68348fc57ee5/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:03.808171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:03.808496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:03.808770 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:03.839514 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] MFA Rules not processed for user `f7f9b61b573243089c5b68348fc57ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:03.840823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:03.846114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:03.907296 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1352/2724] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:03.920950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:03.961442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:03.963593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:04.014352 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71aadd5-a7e2-438c-b9d2-ce7a109333dd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.019731 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.020487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.020780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.021295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.021999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.024790 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.056735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8faee4-f8ff-4ef5-b8b6-b99de435ff72 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.061516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.062163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.062450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.062731 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] PATH_INFO: `/v3/users/d3ddd4abfdbd4c4ca2220d2719700f52` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.063933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ddd4abfdbd4c4ca2220d2719700f52)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.065966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.113961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1373/2725] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:03 2022] POST /identity/v3/projects => generated 378 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:04.118750 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:04.140645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:04.197726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88fdb-d876-4e1c-a46d-e4873a653b6c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.202791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.203415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.203703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.203994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.204695 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.206622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.245632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ddd4abfdbd4c4ca2220d2719700f52'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:04.253145 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ddd4abfdbd4c4ca2220d2719700f52'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:04.253887 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ddd4abfdbd4c4ca2220d2719700f52'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:04.312138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1374/2726] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:04 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:44:04.316365 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:04.318021 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:04.330480 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48f120-39d9-4eca-8071-4c48488e8140 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.336786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.338677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ddd4abfdbd4c4ca2220d2719700f52'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:04.338816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.338918 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ef6b431-31ba-4de9-a27f-769dc4a755c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ddd4abfdbd4c4ca2220d2719700f52'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:04.339032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.339032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.339032 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.340701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.352127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1375/2727] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:04 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:44:04.356699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:04.362880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1353/2728] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:03 2022] DELETE /identity/v3/users/d3ddd4abfdbd4c4ca2220d2719700f52 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:04.368975 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89767640-a25f-4ae3-9bd8-a9183d5e1c45 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.370641 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.370940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.371224 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.391483 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:04.392740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:04.417863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:04.431279 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.431960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.432246 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.432553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] PATH_INFO: `/v3/projects/eb2dfbe3e38d4be0b88570d74b120ebf/users/45f824e15227436f9351882e4850a8d1/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.450919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2dfbe3e38d4be0b88570d74b120ebf, user_id=45f824e15227436f9351882e4850a8d1, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.453590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.470780 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1354/2729] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:04.480714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:04.482809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:04.516953 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1376/2730] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:04 2022] PUT /identity/v3/projects/eb2dfbe3e38d4be0b88570d74b120ebf/users/45f824e15227436f9351882e4850a8d1/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:04.521008 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:04.543650 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6119e-cedf-409b-98a8-7d41d8568af4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.589418 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f349-85ef-480a-b44f-434fa54e5015 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.594843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.595531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.595812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.596087 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.596848 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.598497 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.609997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1377/2731] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:04 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:04.614291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:04.624806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f41ee1-dfa2-4088-b91b-4cc2f9ec7c15 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.629637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64df5c-6a19-4bb8-af75-e0688184ceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.630329 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64df5c-6a19-4bb8-af75-e0688184ceeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.630606 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64df5c-6a19-4bb8-af75-e0688184ceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.630879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64df5c-6a19-4bb8-af75-e0688184ceeb admin admin] PATH_INFO: `/v3/projects/eb2dfbe3e38d4be0b88570d74b120ebf/users/45f824e15227436f9351882e4850a8d1/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.633818 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64df5c-6a19-4bb8-af75-e0688184ceeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2dfbe3e38d4be0b88570d74b120ebf, user_id=45f824e15227436f9351882e4850a8d1, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.636543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64df5c-6a19-4bb8-af75-e0688184ceeb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.648988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1378/2732] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:04 2022] PUT /identity/v3/projects/eb2dfbe3e38d4be0b88570d74b120ebf/users/45f824e15227436f9351882e4850a8d1/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:04.660300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.660645 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.660936 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.668569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:04.692510 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] MFA Rules not processed for user `45f824e15227436f9351882e4850a8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:04.693745 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:04.738533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.739280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.739624 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.739844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.753332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.754915 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:04.765999 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1355/2733] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:04.775607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1379/2734] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:04.787091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:04.789102 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:04.865215 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83cb9f-0540-42ce-ab24-e57da99c88e3 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:04.909822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:04.926968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:04.927738 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:04.928018 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:04.928291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:04.930213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:04.993011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:04.994475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:05.004445 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1356/2735] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:05.269407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:05.269407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:05.352639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd0503-0973-44bf-8b53-7fc7e2aead31 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:05.361577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:05.362243 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:05.364063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:05.364063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] PATH_INFO: `/v3/projects/67bed7dd7b414caeb01b965cb434a681` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:05.364063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67bed7dd7b414caeb01b965cb434a681)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:05.365950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:05.501637 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67bed7dd7b414caeb01b965cb434a681', 'request_id': 'req-3406fb07-9134-4ae0-a4c5-b1247f664504'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.507997 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67bed7dd7b414caeb01b965cb434a681', 'request_id': 'req-3406fb07-9134-4ae0-a4c5-b1247f664504'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.514066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67bed7dd7b414caeb01b965cb434a681', 'request_id': 'req-3406fb07-9134-4ae0-a4c5-b1247f664504'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.574856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] The token cache is being invalidate because project 67bed7dd7b414caeb01b965cb434a681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:05.575155 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.576051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.578184 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1380/2736] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:05 2022] DELETE /identity/v3/projects/67bed7dd7b414caeb01b965cb434a681 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:05.582076 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:05.594684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15213d5-550a-4012-9c29-909b19325a33 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:05.660179 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:05.660874 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:05.661192 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:05.661474 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] PATH_INFO: `/v3/users/e9cae1ab68f54e1a9af3af93d29413f9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:05.662659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cae1ab68f54e1a9af3af93d29413f9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:05.664661 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:05.812352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cae1ab68f54e1a9af3af93d29413f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.820098 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cae1ab68f54e1a9af3af93d29413f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.820818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cae1ab68f54e1a9af3af93d29413f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.882498 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cae1ab68f54e1a9af3af93d29413f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.883091 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cae1ab68f54e1a9af3af93d29413f9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:05.902427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1357/2737] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:05 2022] DELETE /identity/v3/users/e9cae1ab68f54e1a9af3af93d29413f9 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:06.352973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:06.359743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:06.372321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406fb07-9134-4ae0-a4c5-b1247f664504 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:06.387610 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:06.433566 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:06.434222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:06.434506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:06.434787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] PATH_INFO: `/v3/projects/d733c82562104893a1c14ab50f3b1865` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:06.436029 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d733c82562104893a1c14ab50f3b1865)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:06.438886 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:06.543068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd733c82562104893a1c14ab50f3b1865', 'request_id': 'req-d3e068bf-a90f-408c-8d57-e50097f7cb41'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:06.550926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd733c82562104893a1c14ab50f3b1865', 'request_id': 'req-d3e068bf-a90f-408c-8d57-e50097f7cb41'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:06.557968 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd733c82562104893a1c14ab50f3b1865', 'request_id': 'req-d3e068bf-a90f-408c-8d57-e50097f7cb41'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:06.626116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] The token cache is being invalidate because project d733c82562104893a1c14ab50f3b1865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:06.626417 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:06.627091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:06.628925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1381/2738] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:06 2022] DELETE /identity/v3/projects/d733c82562104893a1c14ab50f3b1865 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:06.971981 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:06.973342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:06.993771 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:07.034478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1670c9-fada-4c86-8a49-1645828bd600 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:07.088269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:07.088931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:07.089282 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:07.089534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:07.091402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:07.163696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:07.166292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:07.177057 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1358/2739] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:44:08.743813 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:08.747039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:08.758669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e068bf-a90f-408c-8d57-e50097f7cb41 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:08.764895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:08.765540 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:08.765827 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:08.766104 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:08.769140 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:08.852181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:08.853334 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:08.865308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1382/2740] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:12.603414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:12.604950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:12.662534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438e2dd-9b9f-4cf3-ba69-31bd77b4eb3c service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:12.667221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4592-e953-45da-9050-91bd0b92acd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:12.667839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4592-e953-45da-9050-91bd0b92acd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:12.668127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4592-e953-45da-9050-91bd0b92acd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:12.668418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4592-e953-45da-9050-91bd0b92acd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:12.670527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee4c4592-e953-45da-9050-91bd0b92acd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:12.712231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:12.734827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c4592-e953-45da-9050-91bd0b92acd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:12.736259 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c4592-e953-45da-9050-91bd0b92acd1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:12.747907 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1359/2741] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:13.557937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:13.559401 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:13.573780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f9bb-3381-4175-929c-ce90ac716474 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:13.590143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:13.590143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:13.590143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:13.590143 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] PATH_INFO: `/v3/users/0f5972cbf5a245d583e59a09d509320c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:13.590573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5972cbf5a245d583e59a09d509320c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:13.590573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:13.886324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:13.895127 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5972cbf5a245d583e59a09d509320c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:13.895793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5972cbf5a245d583e59a09d509320c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:13.904016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5972cbf5a245d583e59a09d509320c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:13.921167 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5972cbf5a245d583e59a09d509320c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:14.008736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5972cbf5a245d583e59a09d509320c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:14.016052 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1383/2742] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:13 2022] DELETE /identity/v3/users/0f5972cbf5a245d583e59a09d509320c => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:14.021395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:14.021395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:14.021395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:14.042160 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:14.043491 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:14.111966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1360/2743] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:14.122786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:14.124288 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:14.186719 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3ccfd-90f5-4945-8bf5-3cb008dc8170 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:14.190615 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:14.191258 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:14.191539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:14.191844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:14.202144 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:14.203623 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:14.215321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1384/2744] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:14.733970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:14.735677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:14.750257 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f07bcb-a300-46ec-9706-255692906c95 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:14.755167 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:14.756325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:14.756325 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:14.756654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] PATH_INFO: `/v3/projects/5b03a9ff5cb848ce9c77c39120dc80bb` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:14.757919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b03a9ff5cb848ce9c77c39120dc80bb)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:14.762216 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:14.907979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b03a9ff5cb848ce9c77c39120dc80bb', 'request_id': 'req-7878ef70-bffc-45db-82a2-61e00e46c460'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.059434 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b03a9ff5cb848ce9c77c39120dc80bb', 'request_id': 'req-7878ef70-bffc-45db-82a2-61e00e46c460'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.065632 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b03a9ff5cb848ce9c77c39120dc80bb', 'request_id': 'req-7878ef70-bffc-45db-82a2-61e00e46c460'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.077761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] The token cache is being invalidate because project 5b03a9ff5cb848ce9c77c39120dc80bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:15.078058 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.078756 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.080279 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1361/2745] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:14 2022] DELETE /identity/v3/projects/5b03a9ff5cb848ce9c77c39120dc80bb => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:15.084309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:15.097353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7b948-3698-44d4-9ed1-65ccccf8ff77 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:15.192699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:15.193675 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:15.194206 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:15.194696 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] PATH_INFO: `/v3/users/f7f9b61b573243089c5b68348fc57ee5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:15.197036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f9b61b573243089c5b68348fc57ee5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:15.199767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:15.421215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f9b61b573243089c5b68348fc57ee5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.422294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f9b61b573243089c5b68348fc57ee5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.435796 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f9b61b573243089c5b68348fc57ee5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.439226 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:15.457051 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f9b61b573243089c5b68348fc57ee5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.556730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f9b61b573243089c5b68348fc57ee5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:15.563773 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1385/2746] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:15 2022] DELETE /identity/v3/users/f7f9b61b573243089c5b68348fc57ee5 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:16.079473 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:16.081148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:16.096090 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ef70-bffc-45db-82a2-61e00e46c460 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:16.161847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:16.162500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:16.163004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:16.163292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] PATH_INFO: `/v3/projects/8c06a06e8f34435995766e8973a44262` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:16.165267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c06a06e8f34435995766e8973a44262)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:16.167645 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:16.169572 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:16.311646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c06a06e8f34435995766e8973a44262', 'request_id': 'req-08132d29-f741-4306-a74c-b421b085c92f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.396293 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c06a06e8f34435995766e8973a44262', 'request_id': 'req-08132d29-f741-4306-a74c-b421b085c92f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.404243 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c06a06e8f34435995766e8973a44262', 'request_id': 'req-08132d29-f741-4306-a74c-b421b085c92f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.430778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] The token cache is being invalidate because project 8c06a06e8f34435995766e8973a44262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:16.431240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.431977 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.436674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1362/2747] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:16 2022] DELETE /identity/v3/projects/8c06a06e8f34435995766e8973a44262 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:16.439787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:16.442732 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:16.464196 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc938fc-a1c9-47d6-808e-65aa535fb24e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:16.536851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:16.537669 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:16.538054 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:16.538448 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] PATH_INFO: `/v3/users/45f824e15227436f9351882e4850a8d1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:16.540554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f824e15227436f9351882e4850a8d1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:16.545501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:16.800390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f824e15227436f9351882e4850a8d1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.801373 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f824e15227436f9351882e4850a8d1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.819488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f824e15227436f9351882e4850a8d1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.839720 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f824e15227436f9351882e4850a8d1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.906154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f824e15227436f9351882e4850a8d1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:16.915142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1386/2748] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:16 2022] DELETE /identity/v3/users/45f824e15227436f9351882e4850a8d1 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:17.252908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:17.256366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:17.275467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08132d29-f741-4306-a74c-b421b085c92f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:17.358010 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:17.359013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:17.359414 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:17.360344 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] PATH_INFO: `/v3/users/e2e9553297c842b3a3c308cdd09aad8d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:17.361968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e9553297c842b3a3c308cdd09aad8d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:17.364205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:17.524317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:17.528094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:17.539419 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adff42-d8d9-4c75-a740-e105a2f76a93 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:17.545124 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:17.546078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:17.546407 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:17.546647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] PATH_INFO: `/v3/projects/eb2dfbe3e38d4be0b88570d74b120ebf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:17.547844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2dfbe3e38d4be0b88570d74b120ebf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:17.549823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:17.564990 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e9553297c842b3a3c308cdd09aad8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.573353 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e9553297c842b3a3c308cdd09aad8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.574064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e9553297c842b3a3c308cdd09aad8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.676379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e9553297c842b3a3c308cdd09aad8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.677187 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5bfbbe1b-2d85-4e17-94fd-d72e57b86677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e9553297c842b3a3c308cdd09aad8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.697300 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2dfbe3e38d4be0b88570d74b120ebf', 'request_id': 'req-4d9ab33c-6300-449c-af07-7f96697f656f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.706183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:17.706425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1363/2749] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:17 2022] DELETE /identity/v3/users/e2e9553297c842b3a3c308cdd09aad8d => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:17.706523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2dfbe3e38d4be0b88570d74b120ebf', 'request_id': 'req-4d9ab33c-6300-449c-af07-7f96697f656f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.710421 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2dfbe3e38d4be0b88570d74b120ebf', 'request_id': 'req-4d9ab33c-6300-449c-af07-7f96697f656f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.791089 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] The token cache is being invalidate because project eb2dfbe3e38d4be0b88570d74b120ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:17.791381 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.792033 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4d9ab33c-6300-449c-af07-7f96697f656f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:17.793856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1387/2750] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:17 2022] DELETE /identity/v3/projects/eb2dfbe3e38d4be0b88570d74b120ebf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:17.799024 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f37f0a-a63d-4b7c-b433-013c94d7ada8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:17.799317 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f37f0a-a63d-4b7c-b433-013c94d7ada8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:17.799658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f37f0a-a63d-4b7c-b433-013c94d7ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:17.821306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-04f37f0a-a63d-4b7c-b433-013c94d7ada8 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:17.822541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-04f37f0a-a63d-4b7c-b433-013c94d7ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:17.827174 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:17.887589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1364/2751] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:17.895930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:17.896217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:17.896545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:17.917925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:17.919132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:17.927368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1388/2752] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:17.932766 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f37f0a-a63d-4b7c-b433-013c94d7ada8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:17.944806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f37f0a-a63d-4b7c-b433-013c94d7ada8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:17.948687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:17.949586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:17.952761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:17.953402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:17.953686 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:17.954016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:17.954693 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:44:17.954964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:17.956899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:17.962668 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929839-7841-4a55-a8ed-6d107e001096 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:17.968108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:17.968950 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:17.972916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:17.972916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] PATH_INFO: `/v3/projects/1165baf67df0484dbe4163c5a750784a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:17.972916 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1165baf67df0484dbe4163c5a750784a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:17.973242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1365/2753] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:44:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:44:17.973340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:17.977822 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:17.992961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317458c5-5dff-4d61-98fd-3d445be02945 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:17.998255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.006577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.006577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.006577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.006577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.006577 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.124342 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1366/2754] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:17 2022] POST /identity/v3/projects => generated 392 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:44:18.128775 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.135309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1165baf67df0484dbe4163c5a750784a', 'request_id': 'req-72692dea-0d61-4c67-9226-02c624c54bfe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:18.142434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1165baf67df0484dbe4163c5a750784a', 'request_id': 'req-72692dea-0d61-4c67-9226-02c624c54bfe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:18.149572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1165baf67df0484dbe4163c5a750784a', 'request_id': 'req-72692dea-0d61-4c67-9226-02c624c54bfe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:18.194013 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b09bf0-cf8a-466e-a687-ff6e578f1b6f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.199032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.199658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.206971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.206971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.206971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.206971 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.283425 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] The token cache is being invalidate because project 1165baf67df0484dbe4163c5a750784a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:18.283860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:18.284972 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-72692dea-0d61-4c67-9226-02c624c54bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:18.291791 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1389/2755] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:17 2022] DELETE /identity/v3/projects/1165baf67df0484dbe4163c5a750784a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:18.293888 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ca01-7c49-43ee-8cf0-e9bdb6a49653 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.294559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ca01-7c49-43ee-8cf0-e9bdb6a49653 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.294944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ca01-7c49-43ee-8cf0-e9bdb6a49653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.315317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c0a1ca01-7c49-43ee-8cf0-e9bdb6a49653 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:18.316564 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0a1ca01-7c49-43ee-8cf0-e9bdb6a49653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:18.324549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1390/2756] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:18.332880 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84d54a-a0b9-4679-85e0-700cc88de1cc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.333165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84d54a-a0b9-4679-85e0-700cc88de1cc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.333446 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84d54a-a0b9-4679-85e0-700cc88de1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.351073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1367/2757] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:18 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:44:18.354063 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7f84d54a-a0b9-4679-85e0-700cc88de1cc None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:18.356147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7f84d54a-a0b9-4679-85e0-700cc88de1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:18.359127 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.361028 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:18.364247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1391/2758] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:18.368482 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84d54a-a0b9-4679-85e0-700cc88de1cc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.374341 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed1aa7-7683-42e0-a1a5-9d4a17331cdb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.378125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84d54a-a0b9-4679-85e0-700cc88de1cc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.379119 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.379764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.380049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.380322 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.381051 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.382626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.383098 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.385321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.385619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.385904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.386722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:44:18.387774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.390294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.396171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1368/2759] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:18 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:44:18.399586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.403985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1392/2760] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:44:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:44:18.408339 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.410392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3600242-d78b-46b2-9abb-44da5a9a398f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.415800 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.416569 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.416763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.417030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.417726 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.419586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.419911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace11c6-ffb9-43da-865a-3ab8a848b038 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.424687 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.425303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.425582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.425902 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.426575 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.428487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.431153 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1369/2761] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:18 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:18.435193 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.444884 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d17ba5-bc88-4add-99b0-74889a104f62 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.451036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc85ad-38b7-47de-80c6-e886ea207032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.451719 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc85ad-38b7-47de-80c6-e886ea207032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.452006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc85ad-38b7-47de-80c6-e886ea207032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.452287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc85ad-38b7-47de-80c6-e886ea207032 admin admin] PATH_INFO: `/v3/projects/176dd6e1c7cd40ee91fb2d3eded8b748/users/6bc485ba90294527ab8b511bf82b0fca/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.469332 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc85ad-38b7-47de-80c6-e886ea207032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=176dd6e1c7cd40ee91fb2d3eded8b748, user_id=6bc485ba90294527ab8b511bf82b0fca, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.472023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc85ad-38b7-47de-80c6-e886ea207032 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.535007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1393/2762] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:18 2022] POST /identity/v3/projects => generated 372 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:44:18.539230 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.557899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1370/2763] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:18 2022] PUT /identity/v3/projects/176dd6e1c7cd40ee91fb2d3eded8b748/users/6bc485ba90294527ab8b511bf82b0fca/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:18.569204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.569502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.569782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.599002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] MFA Rules not processed for user `6bc485ba90294527ab8b511bf82b0fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:18.600214 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:18.606501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e82a4-8445-4700-8ac5-6e71833db4cc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.607398 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:18.615930 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:18.667047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.667803 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.668094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.668406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.669413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.671569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.672112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1371/2764] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:18.684168 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.685844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:18.743630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9bddd-ef54-43dd-9969-169042ecf55e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.747409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.748233 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.748545 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.748735 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1394/2765] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:18 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:44:18.755019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.755159 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.757916 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.759291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f60db7-d68c-4355-ac36-d697fff9b606 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.759898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.764135 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.764823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.765108 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.765386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.766132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.768007 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.769405 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1372/2766] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:44:18.778292 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1395/2767] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:18 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:18.782269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.794662 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5b04e-4d57-486b-bde1-450ba7c5e9bc service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.801020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0f02e-6df1-493e-bcdc-b27fa64cee90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.801020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0f02e-6df1-493e-bcdc-b27fa64cee90 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.801020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0f02e-6df1-493e-bcdc-b27fa64cee90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.801020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0f02e-6df1-493e-bcdc-b27fa64cee90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.801467 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0f02e-6df1-493e-bcdc-b27fa64cee90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.803441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0f02e-6df1-493e-bcdc-b27fa64cee90 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.815825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1373/2768] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:18 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:44:18.820255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:18.830303 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e670d-ab64-4728-a418-21828d14d29d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:18.835112 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:18.835777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.836195 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.836490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] PATH_INFO: `/v3/projects/fa7926ba1045431c965f03c2d15b319b/users/dcadd748cf354d6cb75dbe9942d6de29/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.852819 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7926ba1045431c965f03c2d15b319b, user_id=dcadd748cf354d6cb75dbe9942d6de29, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:18.855572 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:18.927994 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1396/2769] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:18 2022] PUT /identity/v3/projects/fa7926ba1045431c965f03c2d15b319b/users/dcadd748cf354d6cb75dbe9942d6de29/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:18.939840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52feb4f-89bb-4074-9357-a877b44f16c4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:18.940130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52feb4f-89bb-4074-9357-a877b44f16c4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:18.940407 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52feb4f-89bb-4074-9357-a877b44f16c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:18.969469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d52feb4f-89bb-4074-9357-a877b44f16c4 None None] MFA Rules not processed for user `dcadd748cf354d6cb75dbe9942d6de29`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:18.970704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d52feb4f-89bb-4074-9357-a877b44f16c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:19.036281 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1374/2770] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:19.049989 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:19.051649 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:19.129432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600289f5-8ebc-4520-9973-efe06fbeb795 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:19.135785 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491a674-71f0-402e-8265-da5fbcd0c7da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:19.136409 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491a674-71f0-402e-8265-da5fbcd0c7da service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:19.137244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491a674-71f0-402e-8265-da5fbcd0c7da service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:19.137244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491a674-71f0-402e-8265-da5fbcd0c7da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:19.145076 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491a674-71f0-402e-8265-da5fbcd0c7da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:19.146680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491a674-71f0-402e-8265-da5fbcd0c7da service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:19.156859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1397/2771] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:24.472384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52feb4f-89bb-4074-9357-a877b44f16c4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:24.502347 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:24.534314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52feb4f-89bb-4074-9357-a877b44f16c4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:24.539162 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:24.540526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:24.540799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:24.541068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] PATH_INFO: `/v3/users/dcadd748cf354d6cb75dbe9942d6de29` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:24.542248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcadd748cf354d6cb75dbe9942d6de29)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:24.544568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:24.736288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcadd748cf354d6cb75dbe9942d6de29'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:24.743878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcadd748cf354d6cb75dbe9942d6de29'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:24.744596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcadd748cf354d6cb75dbe9942d6de29'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:24.818181 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcadd748cf354d6cb75dbe9942d6de29'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:24.818852 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcadd748cf354d6cb75dbe9942d6de29'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:24.838940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1375/2772] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:24 2022] DELETE /identity/v3/users/dcadd748cf354d6cb75dbe9942d6de29 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:24.843554 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:24.844031 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:24.844149 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:24.864512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:24.865881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:24.913166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:24.925730 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1398/2773] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:24.936840 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:24.938250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:25.000222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5e9dd-33fc-48bd-8a15-3bfb95466534 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:25.009496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8677456-b4f3-4079-9a4b-b799009fe6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:25.009496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8677456-b4f3-4079-9a4b-b799009fe6af service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:25.009496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8677456-b4f3-4079-9a4b-b799009fe6af service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:25.009496 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8677456-b4f3-4079-9a4b-b799009fe6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:25.015926 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8677456-b4f3-4079-9a4b-b799009fe6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:25.017389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8677456-b4f3-4079-9a4b-b799009fe6af service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:25.028002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1376/2774] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:25.466832 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:25.468326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:25.480966 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0736cd0-057a-4718-a231-8fb0b9d69d99 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:25.485740 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:25.486393 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:25.486679 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:25.486981 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] PATH_INFO: `/v3/projects/fa7926ba1045431c965f03c2d15b319b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:25.488228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa7926ba1045431c965f03c2d15b319b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:25.490231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:25.592587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa7926ba1045431c965f03c2d15b319b', 'request_id': 'req-591c22b9-17cc-4c23-9d74-cf8362038fc7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:25.599128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa7926ba1045431c965f03c2d15b319b', 'request_id': 'req-591c22b9-17cc-4c23-9d74-cf8362038fc7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:25.605166 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa7926ba1045431c965f03c2d15b319b', 'request_id': 'req-591c22b9-17cc-4c23-9d74-cf8362038fc7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:25.699191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] The token cache is being invalidate because project fa7926ba1045431c965f03c2d15b319b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:25.699551 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:25.700274 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-591c22b9-17cc-4c23-9d74-cf8362038fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:25.701890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1399/2775] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:25 2022] DELETE /identity/v3/projects/fa7926ba1045431c965f03c2d15b319b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:25.706922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac958-0b33-4275-95f4-392510d47270 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:25.707443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac958-0b33-4275-95f4-392510d47270 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:25.707559 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac958-0b33-4275-95f4-392510d47270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:25.727670 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1a8ac958-0b33-4275-95f4-392510d47270 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:25.728902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a8ac958-0b33-4275-95f4-392510d47270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:25.773486 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:25.787574 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1377/2776] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:25.796816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aede1-050a-465d-9490-dd25a36215a1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:25.797100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aede1-050a-465d-9490-dd25a36215a1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:25.797372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aede1-050a-465d-9490-dd25a36215a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:25.819080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7a0aede1-050a-465d-9490-dd25a36215a1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:25.820317 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7a0aede1-050a-465d-9490-dd25a36215a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:25.830577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1400/2777] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:25.834831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ac958-0b33-4275-95f4-392510d47270 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:25.844506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ac958-0b33-4275-95f4-392510d47270 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:25.849221 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:25.849859 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:25.850144 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:25.850418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:25.851122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:44:25.851427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:25.853247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:25.865767 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1378/2778] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:44:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:44:25.873100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0aede1-050a-465d-9490-dd25a36215a1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:25.883026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0aede1-050a-465d-9490-dd25a36215a1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:25.888630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:25.889263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:25.889547 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:25.889852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:25.890545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:25.892569 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.065895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1401/2779] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:25 2022] POST /identity/v3/projects => generated 380 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:26.070418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.081121 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70b689-4dea-422a-846c-0b433805592f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.140669 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.141565 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.142023 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.142426 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.143225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.145269 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.252732 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1379/2780] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:26 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:26.257094 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.268359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eef46-e542-4527-b344-67637ee08741 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.273059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.273683 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.274638 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.274938 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.275753 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.277481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.288686 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1402/2781] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:26 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:26.293195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.303453 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ea2a3-0221-4c07-8456-c3eb79537072 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.308185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60ece5-91aa-457e-9949-dd9105a540a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.308832 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ece5-91aa-457e-9949-dd9105a540a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.309117 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ece5-91aa-457e-9949-dd9105a540a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.309390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ece5-91aa-457e-9949-dd9105a540a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.310085 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60ece5-91aa-457e-9949-dd9105a540a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.311682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60ece5-91aa-457e-9949-dd9105a540a4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.322964 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1380/2782] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:26 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:26.327189 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.338762 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7f986-13e4-4f80-940a-d23c62f82160 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.343354 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.344009 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.344291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.344600 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] PATH_INFO: `/v3/projects/321454b68d904680b4a2499787c30b8a/users/a11e738c8e304976be09ebffcd5cf7d9/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.359811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321454b68d904680b4a2499787c30b8a, user_id=a11e738c8e304976be09ebffcd5cf7d9, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.362539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.464910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1403/2783] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:26 2022] PUT /identity/v3/projects/321454b68d904680b4a2499787c30b8a/users/a11e738c8e304976be09ebffcd5cf7d9/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:26.476602 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ab90b-dfdb-4d8c-9ceb-f5c516dad3ef None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.484694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ab90b-dfdb-4d8c-9ceb-f5c516dad3ef None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.484694 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ab90b-dfdb-4d8c-9ceb-f5c516dad3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.505558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-537ab90b-dfdb-4d8c-9ceb-f5c516dad3ef None None] MFA Rules not processed for user `a11e738c8e304976be09ebffcd5cf7d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:26.506784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-537ab90b-dfdb-4d8c-9ceb-f5c516dad3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:26.574713 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1381/2784] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:26.579217 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.642416 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf18b2-50cc-441d-b7c5-8edb87c637b1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.647293 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.650346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.650346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.650346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.650346 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.651491 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.733071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:26.740181 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1404/2785] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:26 2022] POST /identity/v3/projects => generated 380 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:26.745101 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ab90b-dfdb-4d8c-9ceb-f5c516dad3ef None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.755864 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ab90b-dfdb-4d8c-9ceb-f5c516dad3ef None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.800787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:26.813886 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.814508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.814795 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.815073 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.815743 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.817689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.900734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1382/2786] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:26 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:26.904703 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.915859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39a6d-ba3f-4555-b51c-08edf5e1d475 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.920896 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.921516 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.921802 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.922081 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.922846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.924481 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:26.935718 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1405/2787] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:26 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:26.941286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:26.952660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7456-8add-4a88-bb4d-4318748e289a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:26.957689 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:26.958385 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:26.958654 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:26.958959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] PATH_INFO: `/v3/projects/ffa2a47058994fadb175b969c7988ebe/users/0efde9baaeba4914a7553070e946c8b3/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:26.975318 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa2a47058994fadb175b969c7988ebe, user_id=0efde9baaeba4914a7553070e946c8b3, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:26.978707 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:27.076020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1383/2788] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:26 2022] PUT /identity/v3/projects/ffa2a47058994fadb175b969c7988ebe/users/0efde9baaeba4914a7553070e946c8b3/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:27.080055 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:27.091319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06cd55-3933-4060-92f7-aa5e580410f7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:27.151058 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8a268-7d00-428a-85d4-f80915b29074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:27.151604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8a268-7d00-428a-85d4-f80915b29074 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:27.151890 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8a268-7d00-428a-85d4-f80915b29074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:27.152174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8a268-7d00-428a-85d4-f80915b29074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:27.152926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8a268-7d00-428a-85d4-f80915b29074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:27.154630 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8a268-7d00-428a-85d4-f80915b29074 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:27.166080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1406/2789] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:27 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:27.170870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:27.181609 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876e040-fc0a-4a90-a90e-82837e63991b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:27.186442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:27.187109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:27.187397 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:27.187679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] PATH_INFO: `/v3/projects/ffa2a47058994fadb175b969c7988ebe/users/0efde9baaeba4914a7553070e946c8b3/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:27.189882 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa2a47058994fadb175b969c7988ebe, user_id=0efde9baaeba4914a7553070e946c8b3, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:27.192594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:27.203267 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1384/2790] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:27 2022] PUT /identity/v3/projects/ffa2a47058994fadb175b969c7988ebe/users/0efde9baaeba4914a7553070e946c8b3/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:27.214553 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50f3f8-e082-4295-b5fa-00c8151946de None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:27.214837 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50f3f8-e082-4295-b5fa-00c8151946de None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:27.215122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50f3f8-e082-4295-b5fa-00c8151946de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:27.242905 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2b50f3f8-e082-4295-b5fa-00c8151946de None None] MFA Rules not processed for user `0efde9baaeba4914a7553070e946c8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:27.244174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2b50f3f8-e082-4295-b5fa-00c8151946de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:27.316782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1407/2791] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:27.328404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:27.330113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:27.389927 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1984d3-11eb-4879-bf0d-f7301d3ff210 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:27.393788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:27.394408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:27.394684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:27.394959 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:27.405173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:27.406658 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:27.417785 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1385/2792] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:27.632545 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50f3f8-e082-4295-b5fa-00c8151946de None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:27.642573 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50f3f8-e082-4295-b5fa-00c8151946de None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:27.646180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:27.646787 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:27.647066 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:27.647342 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:27.705492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:27.706861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:27.715965 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1408/2793] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:37.412792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=49,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:37.711655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:42.501794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:42.564244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3e64-0371-4f70-83fb-714c4defda1c service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:42.568868 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:42.569508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:42.569792 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:42.570072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] PATH_INFO: `/v3/users/a11e738c8e304976be09ebffcd5cf7d9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:42.571242 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11e738c8e304976be09ebffcd5cf7d9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:42.573965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:42.580347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:42.581859 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:42.596619 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7a7f4-c8d9-4141-a31a-dcaa8e8ac16b service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:42.601389 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:42.602053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:42.602336 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:42.602673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] PATH_INFO: `/v3/users/6bc485ba90294527ab8b511bf82b0fca` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:42.603901 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc485ba90294527ab8b511bf82b0fca)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:42.605999 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:42.804782 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11e738c8e304976be09ebffcd5cf7d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.812824 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11e738c8e304976be09ebffcd5cf7d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.813533 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11e738c8e304976be09ebffcd5cf7d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.815163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc485ba90294527ab8b511bf82b0fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.815847 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc485ba90294527ab8b511bf82b0fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.823358 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc485ba90294527ab8b511bf82b0fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.835523 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc485ba90294527ab8b511bf82b0fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.886818 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11e738c8e304976be09ebffcd5cf7d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.887441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7a0b847-2fb6-46f4-a68a-94587ffa30b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11e738c8e304976be09ebffcd5cf7d9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.906415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1386/2794] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:42 2022] DELETE /identity/v3/users/a11e738c8e304976be09ebffcd5cf7d9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:42.910742 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:42.911032 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:42.911315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:42.926065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5709c3c-5b5f-4e9a-b307-33059b9e49fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc485ba90294527ab8b511bf82b0fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:42.932155 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:42.933319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1409/2795] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:42 2022] DELETE /identity/v3/users/6bc485ba90294527ab8b511bf82b0fca => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:42.933455 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:42.937874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:42.938447 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:42.938737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:42.958836 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:42.960078 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:43.000861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1387/2796] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:43.011815 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:43.013277 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:43.022699 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1410/2797] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:43.033362 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:43.034804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:43.036504 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:43.072582 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8167a5-f7de-43b2-b6a2-e856f07eedec None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:43.076997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:43.077625 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:43.077912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:43.078188 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:43.088406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:43.089943 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:43.093232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c4be2-dc21-4ae9-80f0-6a0fd56c969a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:43.097203 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:43.097850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:43.098137 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:43.098415 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:43.101130 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1388/2798] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:43.109557 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:43.111128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:43.121228 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1411/2799] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:44:43.625537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:43.627077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:43.640746 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0defee3-dcd5-41c1-a12c-ca375b708d07 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:43.644775 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:43.646433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:43.648967 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:43.649621 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:43.649908 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:43.650191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] PATH_INFO: `/v3/projects/321454b68d904680b4a2499787c30b8a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:43.651508 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=321454b68d904680b4a2499787c30b8a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:43.653610 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:43.661879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04ecad-0a81-485d-8a56-8625094e8ec9 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:43.673587 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:43.674296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:43.674590 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:43.674869 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] PATH_INFO: `/v3/projects/176dd6e1c7cd40ee91fb2d3eded8b748` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:43.676178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] RBAC: Authorizing `identity:delete_project(project_id=176dd6e1c7cd40ee91fb2d3eded8b748)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:43.678223 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:43.690185 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:43.779218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321454b68d904680b4a2499787c30b8a', 'request_id': 'req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.801604 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176dd6e1c7cd40ee91fb2d3eded8b748', 'request_id': 'req-683b5ba5-2042-4320-bee5-8dda1a3d48af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.809085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176dd6e1c7cd40ee91fb2d3eded8b748', 'request_id': 'req-683b5ba5-2042-4320-bee5-8dda1a3d48af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.814851 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176dd6e1c7cd40ee91fb2d3eded8b748', 'request_id': 'req-683b5ba5-2042-4320-bee5-8dda1a3d48af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.862720 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321454b68d904680b4a2499787c30b8a', 'request_id': 'req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.871688 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321454b68d904680b4a2499787c30b8a', 'request_id': 'req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.880225 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] The token cache is being invalidate because project 321454b68d904680b4a2499787c30b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:43.880553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.881202 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.882721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1389/2800] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:43 2022] DELETE /identity/v3/projects/321454b68d904680b4a2499787c30b8a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:43.886920 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:43.888244 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:43.953022 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fe783-50f3-46f2-b2c0-dcfd9993c2e9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:43.955322 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] The token cache is being invalidate because project 176dd6e1c7cd40ee91fb2d3eded8b748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:43.955625 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.956291 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:43.957907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1412/2801] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:43 2022] DELETE /identity/v3/projects/176dd6e1c7cd40ee91fb2d3eded8b748 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:44.019213 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:44.029912 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:44.030584 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:44.030866 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:44.031143 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] PATH_INFO: `/v3/users/0efde9baaeba4914a7553070e946c8b3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:44.032660 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efde9baaeba4914a7553070e946c8b3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:44.034635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:44.199552 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efde9baaeba4914a7553070e946c8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:44.208292 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efde9baaeba4914a7553070e946c8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:44.209701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efde9baaeba4914a7553070e946c8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:44.268762 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efde9baaeba4914a7553070e946c8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:44.269449 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7eb8d7c-3073-480a-b926-5dfdd9e47f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efde9baaeba4914a7553070e946c8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:44.289135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1390/2802] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:43 2022] DELETE /identity/v3/users/0efde9baaeba4914a7553070e946c8b3 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:44:44.722027 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:44.723601 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:44.739036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b5ba5-2042-4320-bee5-8dda1a3d48af admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:44.798607 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:44.799257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:44.799558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:44.799823 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] PATH_INFO: `/v3/projects/ffa2a47058994fadb175b969c7988ebe` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:44.801018 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa2a47058994fadb175b969c7988ebe)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:44.802944 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:44.927534 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa2a47058994fadb175b969c7988ebe', 'request_id': 'req-52be7982-da9a-4768-a877-6e523286ca2f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:44.933898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa2a47058994fadb175b969c7988ebe', 'request_id': 'req-52be7982-da9a-4768-a877-6e523286ca2f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:44.939515 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa2a47058994fadb175b969c7988ebe', 'request_id': 'req-52be7982-da9a-4768-a877-6e523286ca2f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:45.004231 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] The token cache is being invalidate because project ffa2a47058994fadb175b969c7988ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:45.004488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:45.005183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52be7982-da9a-4768-a877-6e523286ca2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:45.006754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1413/2803] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:44 2022] DELETE /identity/v3/projects/ffa2a47058994fadb175b969c7988ebe => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:45.012180 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0eee7c-b3ba-4e27-8c09-5072d7bc64ea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.014064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0eee7c-b3ba-4e27-8c09-5072d7bc64ea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.014064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0eee7c-b3ba-4e27-8c09-5072d7bc64ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.033594 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6d0eee7c-b3ba-4e27-8c09-5072d7bc64ea None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:45.034798 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6d0eee7c-b3ba-4e27-8c09-5072d7bc64ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:45.094643 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1391/2804] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:45.102487 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91622c-75e5-4be3-acda-4f49bd90255d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.102771 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91622c-75e5-4be3-acda-4f49bd90255d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.103050 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91622c-75e5-4be3-acda-4f49bd90255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.123298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ed91622c-75e5-4be3-acda-4f49bd90255d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:45.124526 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ed91622c-75e5-4be3-acda-4f49bd90255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:45.132263 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1414/2805] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:45.136708 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0eee7c-b3ba-4e27-8c09-5072d7bc64ea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:45.146330 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0eee7c-b3ba-4e27-8c09-5072d7bc64ea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:45.151634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:45.152255 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.152571 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.152851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.153556 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:44:45.154063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:45.155466 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:45.167979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1392/2806] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:44:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:44:45.173726 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91622c-75e5-4be3-acda-4f49bd90255d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:45.183895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91622c-75e5-4be3-acda-4f49bd90255d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:45.188933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:45.189542 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.189820 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.190093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.190774 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:45.192706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:45.303689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:45.310247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1415/2807] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:45 2022] POST /identity/v3/projects => generated 364 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:45.315541 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:45.326415 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee983-ada4-4cf8-a64f-74319eb2b355 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:45.389106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:45.389757 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.390048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.390324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.391011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:45.392955 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:45.401019 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:45.511025 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1393/2808] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:45 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:45.515816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:45.527297 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d711798-633c-4452-8832-4e148793e851 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:45.532043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:45.532693 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.532978 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.533255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.533958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:45.535543 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:45.547224 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1416/2809] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:45 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:45.551241 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:45.563063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbffca9-e7cf-4bad-8a5c-2ee4a7b40ef8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:45.567944 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eebee9-8282-4103-872d-2466526a7c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:45.568586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eebee9-8282-4103-872d-2466526a7c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.569991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eebee9-8282-4103-872d-2466526a7c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.569991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eebee9-8282-4103-872d-2466526a7c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.569991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eebee9-8282-4103-872d-2466526a7c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:45.571432 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eebee9-8282-4103-872d-2466526a7c8b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:45.582641 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1394/2810] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:45 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:45.586937 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:45.597186 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517cc57-b4a2-4157-819d-49ca374c9d92 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:45.601830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:45.602490 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.602767 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.603043 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] PATH_INFO: `/v3/projects/574e9d881d254a88a170ad98d05487a5/users/d50f5632d52f4a70a22d896fc1c5c01f/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.618305 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=574e9d881d254a88a170ad98d05487a5, user_id=d50f5632d52f4a70a22d896fc1c5c01f, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:45.621713 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:45.727539 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1417/2811] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:45 2022] PUT /identity/v3/projects/574e9d881d254a88a170ad98d05487a5/users/d50f5632d52f4a70a22d896fc1c5c01f/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:45.738938 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2692a5-04ce-4380-a2fd-7d38cc717542 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.739231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2692a5-04ce-4380-a2fd-7d38cc717542 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.739512 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2692a5-04ce-4380-a2fd-7d38cc717542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.771779 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ef2692a5-04ce-4380-a2fd-7d38cc717542 None None] MFA Rules not processed for user `d50f5632d52f4a70a22d896fc1c5c01f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:45.773043 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ef2692a5-04ce-4380-a2fd-7d38cc717542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:45.854831 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1395/2812] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:45.859508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:45.930754 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda210c8-a85d-4a77-a861-29ed35995d1f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:45.936434 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:45.937072 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:45.937353 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:45.937628 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:45.938302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:45.940212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:46.032034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1418/2813] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:45 2022] POST /identity/v3/projects => generated 366 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:44:46.036050 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2692a5-04ce-4380-a2fd-7d38cc717542 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:46.048863 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2692a5-04ce-4380-a2fd-7d38cc717542 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:46.107829 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:46.108380 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:46.108696 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:46.108975 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:46.109664 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:46.111593 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:46.200612 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1396/2814] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:46 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:46.204812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:46.219209 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277093ff-46f3-49d8-a67e-b03d2931b638 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:46.224183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:46.224852 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:46.225138 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:46.225460 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:46.226219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:46.227995 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:46.240219 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1419/2815] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:46 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:46.244444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:46.256763 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f0bf1-481a-4659-bbd1-93009a75c714 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:46.263135 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:46.263823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:46.264109 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:46.264389 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] PATH_INFO: `/v3/projects/ce096b8e543d44c3a2ef52689655725b/users/fa72157b725449649f61084cc23e0696/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:46.282616 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce096b8e543d44c3a2ef52689655725b, user_id=fa72157b725449649f61084cc23e0696, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:46.285408 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:46.361465 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1397/2816] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:46 2022] PUT /identity/v3/projects/ce096b8e543d44c3a2ef52689655725b/users/fa72157b725449649f61084cc23e0696/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:46.365707 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:46.377555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70afb0-738c-4041-ae2f-273e0a368764 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:46.437020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402ff90-ceb2-4d51-8eb3-f487fa087624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:46.437766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402ff90-ceb2-4d51-8eb3-f487fa087624 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:46.438045 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402ff90-ceb2-4d51-8eb3-f487fa087624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:46.438321 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402ff90-ceb2-4d51-8eb3-f487fa087624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:46.439014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402ff90-ceb2-4d51-8eb3-f487fa087624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:46.440634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402ff90-ceb2-4d51-8eb3-f487fa087624 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:46.451919 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1420/2817] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:46 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:46.456714 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:46.469247 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32646-0a07-414a-ae37-bcd2abdcaee3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:46.474302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:46.474965 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:46.475250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:46.475527 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] PATH_INFO: `/v3/projects/ce096b8e543d44c3a2ef52689655725b/users/fa72157b725449649f61084cc23e0696/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:46.477521 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce096b8e543d44c3a2ef52689655725b, user_id=fa72157b725449649f61084cc23e0696, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:46.480702 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:46.490946 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1398/2818] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:46 2022] PUT /identity/v3/projects/ce096b8e543d44c3a2ef52689655725b/users/fa72157b725449649f61084cc23e0696/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:46.501961 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcb73-9087-4c5f-b1d5-e94baccdc72d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:46.502249 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcb73-9087-4c5f-b1d5-e94baccdc72d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:46.502527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcb73-9087-4c5f-b1d5-e94baccdc72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:46.533309 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4d0bcb73-9087-4c5f-b1d5-e94baccdc72d None None] MFA Rules not processed for user `fa72157b725449649f61084cc23e0696`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:46.534646 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4d0bcb73-9087-4c5f-b1d5-e94baccdc72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:46.559926 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:46.604660 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1421/2819] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:46.616723 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:46.618406 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:46.644080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:46.680315 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec43f51-1692-4a1d-bb22-044731e71a7c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:46.684827 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3abad9-49b3-4ca2-b47b-47aa3992be9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:46.685450 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3abad9-49b3-4ca2-b47b-47aa3992be9e service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:46.685733 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3abad9-49b3-4ca2-b47b-47aa3992be9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:46.686014 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3abad9-49b3-4ca2-b47b-47aa3992be9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:46.695275 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3abad9-49b3-4ca2-b47b-47aa3992be9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:46.696778 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3abad9-49b3-4ca2-b47b-47aa3992be9e service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:46.706870 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1399/2820] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:47.536595 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bcb73-9087-4c5f-b1d5-e94baccdc72d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:47.599130 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bcb73-9087-4c5f-b1d5-e94baccdc72d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:47.603655 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:47.604294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:47.604613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:47.604892 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] PATH_INFO: `/v3/users/d50f5632d52f4a70a22d896fc1c5c01f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:47.606080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d50f5632d52f4a70a22d896fc1c5c01f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:47.608013 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:47.784234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd50f5632d52f4a70a22d896fc1c5c01f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:47.784913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd50f5632d52f4a70a22d896fc1c5c01f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:47.792080 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd50f5632d52f4a70a22d896fc1c5c01f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:47.805233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd50f5632d52f4a70a22d896fc1c5c01f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:47.850903 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd50f5632d52f4a70a22d896fc1c5c01f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:47.857431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1422/2821] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:47 2022] DELETE /identity/v3/users/d50f5632d52f4a70a22d896fc1c5c01f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:47.862000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:47.863169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:47.863169 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:47.881482 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:47.882679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:47.942741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1400/2822] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:47.953188 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:47.954609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:47.998549 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:48.012798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b65db-5ffb-48b1-a46f-7c011fb18c35 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:48.016634 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:48.017260 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:48.017544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:48.017830 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:48.028133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:48.029650 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:48.043377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1423/2823] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:48.479674 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:48.482961 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:48.496422 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465dd0-2d60-4d23-8672-0d2f336aa0d5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:48.502295 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:48.502956 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:48.503237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:48.503516 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] PATH_INFO: `/v3/projects/574e9d881d254a88a170ad98d05487a5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:48.504745 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=574e9d881d254a88a170ad98d05487a5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:48.506648 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:48.624794 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574e9d881d254a88a170ad98d05487a5', 'request_id': 'req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.702288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574e9d881d254a88a170ad98d05487a5', 'request_id': 'req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.708786 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574e9d881d254a88a170ad98d05487a5', 'request_id': 'req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.720596 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] The token cache is being invalidate because project 574e9d881d254a88a170ad98d05487a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:48.720889 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.721530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.723037 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1401/2824] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:48 2022] DELETE /identity/v3/projects/574e9d881d254a88a170ad98d05487a5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:48.727312 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:48.738521 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef60c8-3aba-40f5-a015-9f8697346296 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:48.796900 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:48.797533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:48.797811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:48.798085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] PATH_INFO: `/v3/users/fa72157b725449649f61084cc23e0696` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:48.799232 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa72157b725449649f61084cc23e0696)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:48.801175 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:48.968210 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa72157b725449649f61084cc23e0696'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.968910 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa72157b725449649f61084cc23e0696'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.976247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa72157b725449649f61084cc23e0696'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:48.989117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa72157b725449649f61084cc23e0696'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:49.062174 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e8bd8cc-8a9d-40b6-8c1f-54f01864ed16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa72157b725449649f61084cc23e0696'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:49.068872 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1424/2825] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:44:48 2022] DELETE /identity/v3/users/fa72157b725449649f61084cc23e0696 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:49.516183 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:49.517679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:49.531291 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4df60-9ba8-4551-bb24-8d8c0db5bb3f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:49.593865 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:49.611114 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:49.611788 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:49.612077 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:49.612390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] PATH_INFO: `/v3/projects/ce096b8e543d44c3a2ef52689655725b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:49.613673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce096b8e543d44c3a2ef52689655725b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:49.615629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:49.732088 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce096b8e543d44c3a2ef52689655725b', 'request_id': 'req-144a5d2d-a860-4ce5-9896-8faa8aa136fd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:49.799684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce096b8e543d44c3a2ef52689655725b', 'request_id': 'req-144a5d2d-a860-4ce5-9896-8faa8aa136fd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:49.806113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce096b8e543d44c3a2ef52689655725b', 'request_id': 'req-144a5d2d-a860-4ce5-9896-8faa8aa136fd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:49.818717 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] The token cache is being invalidate because project ce096b8e543d44c3a2ef52689655725b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:44:49.822443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:49.822443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-144a5d2d-a860-4ce5-9896-8faa8aa136fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:44:49.822443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1402/2826] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:44:49 2022] DELETE /identity/v3/projects/ce096b8e543d44c3a2ef52689655725b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:49.826984 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe931a-2e9f-4577-a79e-501ea7c6564b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:49.827469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe931a-2e9f-4577-a79e-501ea7c6564b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:49.827567 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe931a-2e9f-4577-a79e-501ea7c6564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:49.847132 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bbfe931a-2e9f-4577-a79e-501ea7c6564b None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:49.848347 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bbfe931a-2e9f-4577-a79e-501ea7c6564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:49.911593 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1425/2827] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:49.921340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f1061-544d-4411-9f6f-98bf246c4f49 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:49.921630 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f1061-544d-4411-9f6f-98bf246c4f49 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:49.921913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f1061-544d-4411-9f6f-98bf246c4f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:49.943551 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-682f1061-544d-4411-9f6f-98bf246c4f49 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:49.949646 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-682f1061-544d-4411-9f6f-98bf246c4f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:49.960605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1403/2828] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:49.964180 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe931a-2e9f-4577-a79e-501ea7c6564b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:49.969887 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:49.975298 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe931a-2e9f-4577-a79e-501ea7c6564b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:49.982985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:49.982985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:49.982985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:49.982985 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:49.983395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:44:49.983395 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:49.984706 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:49.996313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1426/2829] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:44:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:44:50.001248 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f1061-544d-4411-9f6f-98bf246c4f49 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.010403 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f1061-544d-4411-9f6f-98bf246c4f49 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.016087 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.016734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.017021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.017305 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.018005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.019903 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.087105 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1404/2830] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:50 2022] POST /identity/v3/projects => generated 378 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:50.091295 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.102506 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706d38f-050f-4e20-b769-279c6efbf729 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.166879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.166879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.166879 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.167605 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.167941 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.170472 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.245583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1427/2831] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:50 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:50.250372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.260880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d0ac8-dea6-490f-b899-96244ba9717f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.266352 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.266969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.267271 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.267534 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.268231 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.269957 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.280809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1405/2832] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:50 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:50.284907 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.297255 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1045994-e10c-46f7-a008-3feb64e3d078 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.302172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b1666-d228-43b7-9012-984eb5b995b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.302835 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b1666-d228-43b7-9012-984eb5b995b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.303117 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b1666-d228-43b7-9012-984eb5b995b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.303396 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b1666-d228-43b7-9012-984eb5b995b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.304092 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b1666-d228-43b7-9012-984eb5b995b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.305821 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b1666-d228-43b7-9012-984eb5b995b7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.317993 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1428/2833] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:50 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:50.322372 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.333579 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb7901-f72b-4dbf-a967-76d6937417e6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.338562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.339185 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.339469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.339747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] PATH_INFO: `/v3/projects/f98706b218ac460ea48501e8b2e37fca/users/af3f2decf56b4ed3961e51ce0bdaca75/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.355692 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98706b218ac460ea48501e8b2e37fca, user_id=af3f2decf56b4ed3961e51ce0bdaca75, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.358387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.416931 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1406/2834] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:50 2022] PUT /identity/v3/projects/f98706b218ac460ea48501e8b2e37fca/users/af3f2decf56b4ed3961e51ce0bdaca75/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:50.429954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb416c52-413e-43f6-a6f1-f72f6fa3df32 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.430244 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb416c52-413e-43f6-a6f1-f72f6fa3df32 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.430524 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb416c52-413e-43f6-a6f1-f72f6fa3df32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.463710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cb416c52-413e-43f6-a6f1-f72f6fa3df32 None None] MFA Rules not processed for user `af3f2decf56b4ed3961e51ce0bdaca75`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:50.464960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cb416c52-413e-43f6-a6f1-f72f6fa3df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:50.534059 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1429/2835] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:44:50.538709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.601940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2139ae-825b-4e0c-a5db-b5d5ea964723 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.606629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.607820 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.608165 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.608443 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.609147 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.611067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.691611 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:50.698754 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1407/2836] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:44:50 2022] POST /identity/v3/projects => generated 374 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:44:50.703489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb416c52-413e-43f6-a6f1-f72f6fa3df32 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.714839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb416c52-413e-43f6-a6f1-f72f6fa3df32 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.775671 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:44:50.781934 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.782552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.782833 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.783109 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.783792 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.785755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.909093 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1430/2837] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:44:50 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:44:50.913139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.924481 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eeeff-ceff-47a2-bde9-727c870058de admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.929106 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.929721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.930005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.930313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.931343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.932930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:50.944922 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1408/2838] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:50 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:50.954527 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:50.965198 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0527cb-a52b-4296-ab01-ec19fa9fa8d0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:50.970839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:50.971512 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:50.971793 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:50.972065 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] PATH_INFO: `/v3/projects/d7f08008aa794e3597dafd8566331d46/users/336b13b81fea4c6da5f5019b575ac7b0/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:50.988379 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f08008aa794e3597dafd8566331d46, user_id=336b13b81fea4c6da5f5019b575ac7b0, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:50.991386 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:51.039529 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1431/2839] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:50 2022] PUT /identity/v3/projects/d7f08008aa794e3597dafd8566331d46/users/336b13b81fea4c6da5f5019b575ac7b0/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:51.044012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:51.053923 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad0ce8-f108-4785-93ef-6070172311a9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:51.110226 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec9a1d-7c8c-4333-b6e3-e396ebe14d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:51.110851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec9a1d-7c8c-4333-b6e3-e396ebe14d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:51.111138 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec9a1d-7c8c-4333-b6e3-e396ebe14d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:51.111418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec9a1d-7c8c-4333-b6e3-e396ebe14d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:51.112116 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec9a1d-7c8c-4333-b6e3-e396ebe14d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:51.113752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec9a1d-7c8c-4333-b6e3-e396ebe14d1f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:51.124048 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1409/2840] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:44:51 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:44:51.128327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:51.139071 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3b621-c9b1-4cff-aef1-733f5466b3f7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:51.143697 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:51.144355 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:51.144673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:51.144951 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] PATH_INFO: `/v3/projects/d7f08008aa794e3597dafd8566331d46/users/336b13b81fea4c6da5f5019b575ac7b0/roles/846ff459eaba4359b0bc2e143d793bfd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:51.146913 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f08008aa794e3597dafd8566331d46, user_id=336b13b81fea4c6da5f5019b575ac7b0, role_id=846ff459eaba4359b0bc2e143d793bfd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:51.149586 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:51.159952 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1432/2841] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:44:51 2022] PUT /identity/v3/projects/d7f08008aa794e3597dafd8566331d46/users/336b13b81fea4c6da5f5019b575ac7b0/roles/846ff459eaba4359b0bc2e143d793bfd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:44:51.171139 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:51.171431 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:51.171761 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:51.199506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] MFA Rules not processed for user `336b13b81fea4c6da5f5019b575ac7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:44:51.201511 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:51.283367 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1410/2842] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:44:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:44:51.296617 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:51.298327 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:51.367736 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a929086-1afe-44cf-84d8-792b4f2bdd0a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:51.372390 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:51.373005 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:51.373287 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:51.373560 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:51.432406 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:51.433804 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:51.443283 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1433/2843] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:44:57.648288 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:44:57.649730 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:44:57.662239 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd8a5f-56f6-441f-858f-ed1895088e68 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:44:57.665797 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9e965-84c6-425c-b1df-143a63b3a1c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:44:57.666404 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9e965-84c6-425c-b1df-143a63b3a1c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:44:57.666687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9e965-84c6-425c-b1df-143a63b3a1c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:44:57.667218 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9e965-84c6-425c-b1df-143a63b3a1c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:44:57.676133 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9e965-84c6-425c-b1df-143a63b3a1c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:44:57.677634 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9e965-84c6-425c-b1df-143a63b3a1c0 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:44:57.686992 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1411/2844] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:44:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:01.439118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:04.754681 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:04.766805 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777135df-e25e-4de4-9eaa-8e5417be2d4c service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:04.770672 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:04.771310 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:04.771594 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:04.771866 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:04.773757 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:04.837958 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:04.839452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:04.850285 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1434/2845] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:45:07.682980 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=75 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:14.845596 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=28 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:18.614530 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:18.617311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:18.617311 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:18.641112 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] MFA Rules not processed for user `39f0a10b47b84bd1bd8ff8177362d791`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:18.642387 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:18.650000 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1412/2846] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:45:18.695624 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:18.709083 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae349a1-ddcd-4fbc-850c-d6de41cfdec9 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:18.713026 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:18.713652 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:18.713940 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:18.714222 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:18.716238 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:18.808318 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:18.809967 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:18.825574 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1435/2847] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:19.931148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:19.932558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:20.004921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4890f1-aeaa-4770-9811-b837ec133bcc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:20.010384 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2119d-709f-45c2-9997-6ad207d7f1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:20.010947 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2119d-709f-45c2-9997-6ad207d7f1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:20.011227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2119d-709f-45c2-9997-6ad207d7f1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:20.011502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2119d-709f-45c2-9997-6ad207d7f1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:20.013302 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3aa2119d-709f-45c2-9997-6ad207d7f1d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:20.035842 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2119d-709f-45c2-9997-6ad207d7f1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:20.037313 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2119d-709f-45c2-9997-6ad207d7f1d9 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:20.050395 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1413/2848] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:27.243744 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:27.308743 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83dd58-bc78-4cea-bca5-3fe995c90f84 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:27.315068 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:27.318807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:27.319128 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:27.319423 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] PATH_INFO: `/v3/users/af3f2decf56b4ed3961e51ce0bdaca75` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:27.320797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3f2decf56b4ed3961e51ce0bdaca75)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:27.322777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:27.555213 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3f2decf56b4ed3961e51ce0bdaca75'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:27.555874 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3f2decf56b4ed3961e51ce0bdaca75'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:27.564139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3f2decf56b4ed3961e51ce0bdaca75'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:27.579464 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3f2decf56b4ed3961e51ce0bdaca75'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:27.677179 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3f2decf56b4ed3961e51ce0bdaca75'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:27.683856 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1436/2849] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:45:27 2022] DELETE /identity/v3/users/af3f2decf56b4ed3961e51ce0bdaca75 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:27.688454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:27.688940 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:27.689054 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:27.708899 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:27.710152 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:27.778553 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1414/2850] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:27.789164 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:27.791253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:27.804877 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:27.857294 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa68d02-cc05-4f99-ba97-36b0421e66c2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:27.861474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:27.862099 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:27.862384 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:27.862661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:27.872642 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:27.876710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:27.884404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1437/2851] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:28.284997 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:28.295427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:28.301108 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf667b-8822-4492-86dd-cdc67cdcd3fb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:28.306199 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:28.306873 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:28.307136 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:28.307413 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] PATH_INFO: `/v3/projects/f98706b218ac460ea48501e8b2e37fca` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:28.308605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f98706b218ac460ea48501e8b2e37fca)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:28.310590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:28.404913 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f98706b218ac460ea48501e8b2e37fca', 'request_id': 'req-d89622d6-95c3-46b5-9904-0a97e6e2055a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.454933 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f98706b218ac460ea48501e8b2e37fca', 'request_id': 'req-d89622d6-95c3-46b5-9904-0a97e6e2055a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.462118 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f98706b218ac460ea48501e8b2e37fca', 'request_id': 'req-d89622d6-95c3-46b5-9904-0a97e6e2055a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.475049 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] The token cache is being invalidate because project f98706b218ac460ea48501e8b2e37fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:45:28.475334 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.476012 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.477563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1415/2852] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:45:28 2022] DELETE /identity/v3/projects/f98706b218ac460ea48501e8b2e37fca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:28.481728 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:28.492548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c64e7f-80c0-482d-9220-0aa064896dd2 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:28.551014 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:28.551694 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:28.551942 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:28.552220 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] PATH_INFO: `/v3/users/336b13b81fea4c6da5f5019b575ac7b0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:28.553441 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] RBAC: Authorizing `identity:delete_user(user_id=336b13b81fea4c6da5f5019b575ac7b0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:28.555377 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:28.708621 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336b13b81fea4c6da5f5019b575ac7b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.709316 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336b13b81fea4c6da5f5019b575ac7b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.716492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336b13b81fea4c6da5f5019b575ac7b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.733085 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336b13b81fea4c6da5f5019b575ac7b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.802492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7538b1d8-6402-4c08-87ef-fab496bc033c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336b13b81fea4c6da5f5019b575ac7b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:28.809030 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1438/2853] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:45:28 2022] DELETE /identity/v3/users/336b13b81fea4c6da5f5019b575ac7b0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:29.192492 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:29.194007 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:29.209629 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89622d6-95c3-46b5-9904-0a97e6e2055a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:29.228663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:29.269331 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:29.270011 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:29.270234 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:29.270547 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] PATH_INFO: `/v3/projects/d7f08008aa794e3597dafd8566331d46` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:29.271830 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f08008aa794e3597dafd8566331d46)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:29.274040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:29.411902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f08008aa794e3597dafd8566331d46', 'request_id': 'req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:29.511171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f08008aa794e3597dafd8566331d46', 'request_id': 'req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:29.517861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f08008aa794e3597dafd8566331d46', 'request_id': 'req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:29.529659 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] The token cache is being invalidate because project d7f08008aa794e3597dafd8566331d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:45:29.529902 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:29.530555 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a7c8459-90ec-4d60-ab2a-f511f86f70bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:29.532950 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1416/2854] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:45:29 2022] DELETE /identity/v3/projects/d7f08008aa794e3597dafd8566331d46 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:29.538959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042c9dd-b83e-494d-856b-e1e82d9e95ca None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:29.539468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042c9dd-b83e-494d-856b-e1e82d9e95ca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:29.539588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042c9dd-b83e-494d-856b-e1e82d9e95ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:29.559165 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1042c9dd-b83e-494d-856b-e1e82d9e95ca None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:29.560387 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1042c9dd-b83e-494d-856b-e1e82d9e95ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:29.582370 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:29.620588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1439/2855] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:29.629484 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bc2be-a2e3-4e2b-9675-8ec5b2714eba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:29.629851 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bc2be-a2e3-4e2b-9675-8ec5b2714eba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:29.630113 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bc2be-a2e3-4e2b-9675-8ec5b2714eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:29.650764 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9a6bc2be-a2e3-4e2b-9675-8ec5b2714eba None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:29.651970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9a6bc2be-a2e3-4e2b-9675-8ec5b2714eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:29.660272 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1417/2856] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:29.665000 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042c9dd-b83e-494d-856b-e1e82d9e95ca None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:29.675319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042c9dd-b83e-494d-856b-e1e82d9e95ca None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:29.681075 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:29.681800 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:29.682147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:29.682519 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:29.683313 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:45:29.683780 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:29.685544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:29.698053 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1440/2857] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:45:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:45:29.702590 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bc2be-a2e3-4e2b-9675-8ec5b2714eba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:29.712777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bc2be-a2e3-4e2b-9675-8ec5b2714eba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:29.717444 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:29.718062 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:29.718396 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:29.718679 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:29.719463 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:29.721420 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:29.824375 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1418/2858] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:45:29 2022] POST /identity/v3/projects => generated 374 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:45:29.829474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:29.839568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af55d6-f91e-4633-aa06-8acd8f5b30d7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:29.897508 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:29.898911 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:29.899326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:29.899608 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:29.900577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:29.902843 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:30.004488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1441/2859] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:45:29 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:45:30.009104 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:30.020286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e7944-3d12-4e79-b227-ec52889843d1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:30.024888 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:30.025506 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.025787 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.026061 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.026760 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:30.028377 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:30.039265 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1419/2860] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:45:30 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:45:30.043797 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:30.063469 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f07b55-2fea-4938-a6ba-1ee28a4431ae admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:30.068439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d897c-70d7-4980-a122-b08f6f9dcaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:30.068898 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d897c-70d7-4980-a122-b08f6f9dcaff admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.069183 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d897c-70d7-4980-a122-b08f6f9dcaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.069538 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d897c-70d7-4980-a122-b08f6f9dcaff admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.070372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d897c-70d7-4980-a122-b08f6f9dcaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:30.071990 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d897c-70d7-4980-a122-b08f6f9dcaff admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:30.084003 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1442/2861] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:45:30 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:45:30.087740 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:30.098289 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d2968-5917-4bc2-abc5-2b51b151beda admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:30.102952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:30.103681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.104021 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.104250 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] PATH_INFO: `/v3/projects/d2cd20df794242918858f5284e50c3a1/users/0bf7a0b0ffa54b768bcff38af7d297bd/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.120849 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2cd20df794242918858f5284e50c3a1, user_id=0bf7a0b0ffa54b768bcff38af7d297bd, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:30.123502 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:30.180379 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1420/2862] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:45:30 2022] PUT /identity/v3/projects/d2cd20df794242918858f5284e50c3a1/users/0bf7a0b0ffa54b768bcff38af7d297bd/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:30.191531 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6277c-4a4b-45bf-89d9-0f5b7867eb97 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.191970 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6277c-4a4b-45bf-89d9-0f5b7867eb97 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.193653 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6277c-4a4b-45bf-89d9-0f5b7867eb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.225788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-94b6277c-4a4b-45bf-89d9-0f5b7867eb97 None None] MFA Rules not processed for user `0bf7a0b0ffa54b768bcff38af7d297bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:30.227133 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-94b6277c-4a4b-45bf-89d9-0f5b7867eb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:30.299440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1443/2863] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:30.310701 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:30.312979 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:30.356253 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:30.371682 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d10f4-7984-4c9c-9678-0021ebae3a87 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:30.375921 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d6417-8ae8-4053-b129-84f9d6e39910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:30.376615 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d6417-8ae8-4053-b129-84f9d6e39910 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.376898 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d6417-8ae8-4053-b129-84f9d6e39910 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.377204 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d6417-8ae8-4053-b129-84f9d6e39910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.385954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d6417-8ae8-4053-b129-84f9d6e39910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:30.387456 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d6417-8ae8-4053-b129-84f9d6e39910 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:30.398270 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1421/2864] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:30.424147 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6277c-4a4b-45bf-89d9-0f5b7867eb97 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:30.449960 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:30.491325 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6277c-4a4b-45bf-89d9-0f5b7867eb97 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:30.496271 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:30.497016 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.497308 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.497588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] PATH_INFO: `/v3/users/0bf7a0b0ffa54b768bcff38af7d297bd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.498850 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf7a0b0ffa54b768bcff38af7d297bd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:30.501376 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:30.642844 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf7a0b0ffa54b768bcff38af7d297bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:30.643613 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf7a0b0ffa54b768bcff38af7d297bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:30.652302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf7a0b0ffa54b768bcff38af7d297bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:30.666157 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf7a0b0ffa54b768bcff38af7d297bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:30.734242 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf7a0b0ffa54b768bcff38af7d297bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:30.740943 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1444/2865] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:45:30 2022] DELETE /identity/v3/users/0bf7a0b0ffa54b768bcff38af7d297bd => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:30.745752 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.746047 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.746333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.767171 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:30.768393 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:30.830390 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1422/2866] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:30.840125 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:30.841581 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:30.900666 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839b53-93ea-4042-94bb-1c4c9dcd80fa admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:30.904214 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b91177-49bd-45c2-b0a5-c81d2f1e4234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:30.904897 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91177-49bd-45c2-b0a5-c81d2f1e4234 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:30.909651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91177-49bd-45c2-b0a5-c81d2f1e4234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:30.909651 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91177-49bd-45c2-b0a5-c81d2f1e4234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:30.915544 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b91177-49bd-45c2-b0a5-c81d2f1e4234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:30.917082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b91177-49bd-45c2-b0a5-c81d2f1e4234 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:30.928884 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1445/2867] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:31.407066 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:31.408727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:31.423605 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c6977-a9ce-475a-ad81-bbed74f5d67f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:31.435002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:31.435002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:31.435002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:31.435002 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] PATH_INFO: `/v3/projects/d2cd20df794242918858f5284e50c3a1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:31.435490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2cd20df794242918858f5284e50c3a1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:31.435490 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:31.550131 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2cd20df794242918858f5284e50c3a1', 'request_id': 'req-7700466d-4180-4eb8-9af3-6b45f4c9a394'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:31.615340 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2cd20df794242918858f5284e50c3a1', 'request_id': 'req-7700466d-4180-4eb8-9af3-6b45f4c9a394'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:31.622042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2cd20df794242918858f5284e50c3a1', 'request_id': 'req-7700466d-4180-4eb8-9af3-6b45f4c9a394'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:31.636441 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] The token cache is being invalidate because project d2cd20df794242918858f5284e50c3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:45:31.636772 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:31.637475 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7700466d-4180-4eb8-9af3-6b45f4c9a394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:45:31.638991 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1423/2868] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:45:31 2022] DELETE /identity/v3/projects/d2cd20df794242918858f5284e50c3a1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:31.645319 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d08ea-3570-4e1e-93e3-f8706e8b8f6c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:31.645858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d08ea-3570-4e1e-93e3-f8706e8b8f6c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:31.646034 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d08ea-3570-4e1e-93e3-f8706e8b8f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:31.669676 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-804d08ea-3570-4e1e-93e3-f8706e8b8f6c None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:31.670761 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-804d08ea-3570-4e1e-93e3-f8706e8b8f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:31.724861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:31.747492 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1446/2869] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:31.757727 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d10136-13a5-423c-925a-9f58918497b1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:31.758016 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d10136-13a5-423c-925a-9f58918497b1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:31.758298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d10136-13a5-423c-925a-9f58918497b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:31.782952 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-09d10136-13a5-423c-925a-9f58918497b1 None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:31.784919 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-09d10136-13a5-423c-925a-9f58918497b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:31.796655 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1424/2870] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:31.801191 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d08ea-3570-4e1e-93e3-f8706e8b8f6c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:31.816413 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d08ea-3570-4e1e-93e3-f8706e8b8f6c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:31.824086 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:31.825498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:31.825498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:31.825498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:31.832036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:45:31.832036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:31.832036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:31.849069 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1447/2871] 10.222.0.128 () {58 vars in 1158 bytes} [Tue Jul 26 15:45:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:45:31.850256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d10136-13a5-423c-925a-9f58918497b1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:31.865097 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d10136-13a5-423c-925a-9f58918497b1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:31.870333 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:31.870966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:31.871256 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:31.871537 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:31.872575 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:31.874141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:31.954361 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1425/2872] 10.222.0.128 () {60 vars in 1158 bytes} [Tue Jul 26 15:45:31 2022] POST /identity/v3/projects => generated 384 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:45:31.958555 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:31.969429 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09af9f1-fbe6-428d-b577-3eb7fe7e04ec admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:32.041680 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:32.043558 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:32.043840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:32.044119 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:32.044839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:32.047202 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:32.158969 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1448/2873] 10.222.0.128 () {60 vars in 1149 bytes} [Tue Jul 26 15:45:31 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:45:32.163499 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:32.175825 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ad6e3-dbb6-46e9-80b0-a12d656a20dc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:32.181410 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:32.182040 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:32.182366 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:32.182603 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:32.183350 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:32.185065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:32.195994 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1426/2874] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:45:32 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:45:32.200828 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:32.211268 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc179e-4083-4588-b56a-c5171bd5186a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:32.216588 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970ba5a-e382-4f47-91ca-e1f3cdfe0cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:32.217215 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970ba5a-e382-4f47-91ca-e1f3cdfe0cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:32.217501 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970ba5a-e382-4f47-91ca-e1f3cdfe0cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:32.217782 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970ba5a-e382-4f47-91ca-e1f3cdfe0cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:32.219340 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970ba5a-e382-4f47-91ca-e1f3cdfe0cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:32.221118 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970ba5a-e382-4f47-91ca-e1f3cdfe0cb9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:32.233199 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1449/2875] 10.222.0.128 () {58 vars in 1127 bytes} [Tue Jul 26 15:45:32 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:45:32.238698 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:32.250110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5c166-78be-45bb-a24b-6a02e4faf30b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:32.254880 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:32.255543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:32.255828 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:32.256110 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] PATH_INFO: `/v3/projects/c3d0ec4c669b4f6d8bc80aa8d2fab87f/users/236277d969f644de834cc3f61510c313/roles/dc7c0f1a0476470daff325a7c7adf5f2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:32.275510 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d0ec4c669b4f6d8bc80aa8d2fab87f, user_id=236277d969f644de834cc3f61510c313, role_id=dc7c0f1a0476470daff325a7c7adf5f2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:32.279191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:32.381446 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:32.386355 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1427/2876] 10.222.0.128 () {60 vars in 1488 bytes} [Tue Jul 26 15:45:32 2022] PUT /identity/v3/projects/c3d0ec4c669b4f6d8bc80aa8d2fab87f/users/236277d969f644de834cc3f61510c313/roles/dc7c0f1a0476470daff325a7c7adf5f2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:45:32.392146 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9381e5-e7d0-446f-ac4b-45d6f3e41a0d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:32.392433 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9381e5-e7d0-446f-ac4b-45d6f3e41a0d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:32.392737 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9381e5-e7d0-446f-ac4b-45d6f3e41a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:32.415620 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9d9381e5-e7d0-446f-ac4b-45d6f3e41a0d None None] MFA Rules not processed for user `7e2685b786dd4d2eb4cf48fedfc3b189`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:32.419824 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9d9381e5-e7d0-446f-ac4b-45d6f3e41a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:32.486766 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1450/2877] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:45:32.497383 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:32.500082 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:32.573080 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b57b18-1338-4637-8cdb-6892475850c2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:32.578562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:32.579286 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:32.579801 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:32.580324 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:32.592741 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:32.596409 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:32.606985 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1428/2878] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:34.094037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51388-42db-4c14-98e9-3816a5fe0c47 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:34.094037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51388-42db-4c14-98e9-3816a5fe0c47 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:34.094037 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51388-42db-4c14-98e9-3816a5fe0c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:34.133247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c9b51388-42db-4c14-98e9-3816a5fe0c47 None None] MFA Rules not processed for user `236277d969f644de834cc3f61510c313`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:34.134483 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c9b51388-42db-4c14-98e9-3816a5fe0c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:34.178812 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:34.208881 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1451/2879] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:34.221006 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:34.225428 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:34.316146 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0166-d807-46a3-a21c-6c8c64b2fc7b service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:34.319930 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff6f527aa364534b2fdd0b770924e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:34.320613 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:34.320905 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:34.321186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:34.331314 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:34.332909 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:34.343583 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1429/2880] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:36.134748 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e960d-08b7-4d3f-804c-0791fc9bb0df None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:36.135296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e960d-08b7-4d3f-804c-0791fc9bb0df None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:36.135296 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e960d-08b7-4d3f-804c-0791fc9bb0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:36.154326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7c0e960d-08b7-4d3f-804c-0791fc9bb0df None None] MFA Rules not processed for user `236277d969f644de834cc3f61510c313`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:45:36.156067 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7c0e960d-08b7-4d3f-804c-0791fc9bb0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:36.164156 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1452/2881] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:45:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:45:39.217614 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:45:39.219687 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:39.231813 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e702e00-26c5-4dfb-ade6-20708df72b73 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:45:39.236418 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:45:39.237065 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:45:39.237355 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:45:39.237641 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:45:39.239460 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:45:39.310996 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:45:39.312517 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:45:39.323107 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1430/2882] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:45:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:45:46.149047 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=12 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:45:49.329163 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=82 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:46:19.285704 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e960d-08b7-4d3f-804c-0791fc9bb0df None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:46:19.303918 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e960d-08b7-4d3f-804c-0791fc9bb0df None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:46:19.308908 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:46:19.309552 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:46:19.309839 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:46:19.310116 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:46:19.312040 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:19.325247 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:46:19.328170 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:46:19.348788 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1453/2883] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:46:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:46:24.466735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:46:24.475681 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7228f8-696b-4ed8-9a37-b5c7047eb48b service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:46:24.479635 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:46:24.480260 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:46:24.480568 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:46:24.480846 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:46:24.482711 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:24.492799 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:46:24.494709 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:46:24.502777 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1431/2884] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:46:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:46:24.918829 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:46:24.920689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:24.994858 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185343e4-6330-4501-9924-d0537a07a0e7 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:46:25.000039 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:46:25.000689 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:46:25.000973 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:46:25.001253 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:46:25.002988 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:25.015315 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:46:25.016959 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:46:25.027548 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1454/2885] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:46:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:46:27.875810 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:46:27.885570 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:27.955735 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee757c00-4b19-411c-a4b4-747b9a3c663e service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:46:27.962500 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf316a4-9e37-4e48-8d08-8f68e9c4c225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:46:27.963823 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf316a4-9e37-4e48-8d08-8f68e9c4c225 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:46:27.964063 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf316a4-9e37-4e48-8d08-8f68e9c4c225 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:46:27.964343 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf316a4-9e37-4e48-8d08-8f68e9c4c225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:46:27.966184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-daf316a4-9e37-4e48-8d08-8f68e9c4c225 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:27.978673 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf316a4-9e37-4e48-8d08-8f68e9c4c225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:46:27.980427 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf316a4-9e37-4e48-8d08-8f68e9c4c225 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:46:27.991471 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1432/2886] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:46:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:46:35.022533 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=38 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:46:36.657474 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:46:36.671895 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207fca4-049f-4b03-9bbd-6c9170d05321 service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:46:36.677591 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:46:36.678520 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:46:36.684233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:46:36.684233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:46:36.684233 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:36.694806 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:46:36.696431 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:46:36.708010 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1455/2887] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:46:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:46:37.986968 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=40 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:46:46.703212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:46:48.891208 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:46:48.891739 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:46:48.891854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:46:48.911399 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] MFA Rules not processed for user `236277d969f644de834cc3f61510c313`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:46:48.914821 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:48.919848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1433/2888] 10.222.0.128 () {58 vars in 963 bytes} [Tue Jul 26 15:46:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:46:49.154809 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:46:49.164933 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e8225-5230-47c6-a1e3-082910e28953 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:46:49.168932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:46:49.169577 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:46:49.169860 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:46:49.170139 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:46:49.171932 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:46:49.181816 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:46:49.183273 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:46:49.191380 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1456/2889] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:46:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:46:58.905111 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:46:59.187598 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:47:24.645004 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:24.647604 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:24.666172 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69deefa-eb29-4194-8d06-bd88c98426ea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:24.671878 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e40a33db1a41d59d9a7a3bef5f3ece', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:24.672626 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:24.673030 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:24.673309 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:24.675312 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:24.689715 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:24.692558 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:24.705020 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1434/2890] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:47:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:47:34.700695 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:47:44.603212 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:44.612710 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002e008-08ee-48cd-8c90-8a14ac0082b4 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:44.616498 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:44.617692 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:44.617980 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:44.618257 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:44.620488 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:44.636254 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:44.639372 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:44.650495 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1457/2891] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:47:47.975479 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:47.978639 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:47.991195 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3443c-7d32-4a98-a050-7489afbe3143 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:47.997987 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:47.998683 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:47.998970 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:48.000140 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:48.001339 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:48.014562 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:48.019369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:48.026652 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1435/2892] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:47:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:47:51.820173 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:51.821674 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:51.836145 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee2064-130b-4f3e-bc37-1aed71878075 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:51.842302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:51.842947 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:51.843234 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:51.843517 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] PATH_INFO: `/v3/users/39f0a10b47b84bd1bd8ff8177362d791` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:51.844723 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f0a10b47b84bd1bd8ff8177362d791)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:51.846701 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:52.015105 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f0a10b47b84bd1bd8ff8177362d791'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.015622 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f0a10b47b84bd1bd8ff8177362d791'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.023328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f0a10b47b84bd1bd8ff8177362d791'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.040036 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f0a10b47b84bd1bd8ff8177362d791'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.121755 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f0a10b47b84bd1bd8ff8177362d791'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.132178 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1458/2893] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:47:51 2022] DELETE /identity/v3/users/39f0a10b47b84bd1bd8ff8177362d791 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:47:52.425148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:52.438273 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2098d-29ec-4816-8a10-ec2e747f4efb service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:52.501201 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:52.501809 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:52.502141 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:52.502425 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] PATH_INFO: `/v3/projects/3e5dd905492343709b710fabd8a3d24f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:52.503589 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5dd905492343709b710fabd8a3d24f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:52.505563 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:52.637129 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5dd905492343709b710fabd8a3d24f', 'request_id': 'req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.738951 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5dd905492343709b710fabd8a3d24f', 'request_id': 'req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.745186 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5dd905492343709b710fabd8a3d24f', 'request_id': 'req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.756951 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] The token cache is being invalidate because project 3e5dd905492343709b710fabd8a3d24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:47:52.757238 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.757876 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:52.759402 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1436/2894] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:47:52 2022] DELETE /identity/v3/projects/3e5dd905492343709b710fabd8a3d24f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:47:52.763629 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:52.765102 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:52.779532 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f6a18-ee75-4486-8288-47f20bc65848 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:52.837096 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:52.838082 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:52.838371 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:52.838647 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] PATH_INFO: `/v3/users/7c87d7f195c543d58ea5975f059f262b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:52.839945 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c87d7f195c543d58ea5975f059f262b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:52.842609 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:53.015777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c87d7f195c543d58ea5975f059f262b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.016368 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c87d7f195c543d58ea5975f059f262b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.023111 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c87d7f195c543d58ea5975f059f262b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.027432 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:47:53.035807 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c87d7f195c543d58ea5975f059f262b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.110727 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c87d7f195c543d58ea5975f059f262b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.117148 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1459/2895] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:47:52 2022] DELETE /identity/v3/users/7c87d7f195c543d58ea5975f059f262b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:47:53.403588 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:53.405100 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:53.421848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8f54f-aedc-4f6c-8b12-af62e631caaf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:53.492847 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:53.493503 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:53.493789 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:53.494068 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] PATH_INFO: `/v3/projects/b445144679f543c9af86651752925487` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:53.495240 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b445144679f543c9af86651752925487)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:53.497182 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:53.519470 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:47:53.613861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b445144679f543c9af86651752925487', 'request_id': 'req-6456501d-c423-4c8b-8e13-0194c72621c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.664627 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b445144679f543c9af86651752925487', 'request_id': 'req-6456501d-c423-4c8b-8e13-0194c72621c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.671697 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b445144679f543c9af86651752925487', 'request_id': 'req-6456501d-c423-4c8b-8e13-0194c72621c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.683854 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] The token cache is being invalidate because project b445144679f543c9af86651752925487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:47:53.684177 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.690079 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:53.691637 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1437/2896] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:47:53 2022] DELETE /identity/v3/projects/b445144679f543c9af86651752925487 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:47:55.328537 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:55.330006 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:55.344436 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0ef76-bbc8-45d6-87bb-c70c0a17766d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:55.403507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:55.404207 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:55.404513 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:55.404798 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] PATH_INFO: `/v3/users/236277d969f644de834cc3f61510c313` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:55.405964 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236277d969f644de834cc3f61510c313)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:55.407927 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:55.559568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236277d969f644de834cc3f61510c313'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:55.560324 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236277d969f644de834cc3f61510c313'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:55.568150 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236277d969f644de834cc3f61510c313'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:55.583702 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236277d969f644de834cc3f61510c313'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:55.676100 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236277d969f644de834cc3f61510c313'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:55.682439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1460/2897] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:47:55 2022] DELETE /identity/v3/users/236277d969f644de834cc3f61510c313 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:47:55.934710 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:47:55.936227 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:47:55.951356 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456501d-c423-4c8b-8e13-0194c72621c3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:47:56.017042 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:47:56.017721 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:47:56.018005 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:47:56.018280 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] PATH_INFO: `/v3/projects/c3d0ec4c669b4f6d8bc80aa8d2fab87f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:47:56.019586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d0ec4c669b4f6d8bc80aa8d2fab87f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:47:56.021518 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:47:56.186595 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d0ec4c669b4f6d8bc80aa8d2fab87f', 'request_id': 'req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:56.247442 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d0ec4c669b4f6d8bc80aa8d2fab87f', 'request_id': 'req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:56.254345 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d0ec4c669b4f6d8bc80aa8d2fab87f', 'request_id': 'req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:56.265454 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] The token cache is being invalidate because project c3d0ec4c669b4f6d8bc80aa8d2fab87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:47:56.265747 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:56.266392 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:47:56.267936 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1438/2898] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:47:55 2022] DELETE /identity/v3/projects/c3d0ec4c669b4f6d8bc80aa8d2fab87f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:48:05.679811 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=13,keystone:SELECT=59,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:48:06.262388 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:48:14.336332 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:14.338020 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:14.397435 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e084320-3eeb-4353-832b-3834c9a22e06 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:14.448489 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:14.449154 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:14.449440 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:14.449717 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:14.451583 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:14.515091 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:14.516576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:14.525777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1461/2899] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:48:20.778148 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:20.779675 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:20.795861 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd722a-2f1d-4d3b-ac62-589d4fb398c9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:20.799839 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ecc741022f4d6189f4d3f19b4cdbe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:20.800488 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:20.800776 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:20.801546 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:20.803064 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:20.898235 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:20.899684 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:20.914036 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1439/2900] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:48:21.505439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:21.506885 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:21.578777 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6b64e-15a5-4e44-8b99-a3f781f11fd7 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:21.583925 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:21.584576 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:21.584861 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:21.585142 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:21.586957 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:21.599328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:21.600955 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:21.607326 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:48:21.612302 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1462/2901] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:48:27.669663 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:27.671128 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:27.687067 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8848a-c0c7-4aa1-8040-a800dfec7b1b service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:27.692205 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:27.692844 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:27.693122 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:27.693400 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:27.695306 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:27.708184 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:27.709966 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:27.721034 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1440/2902] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:48:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:48:27.803867 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:27.805507 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:27.819722 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe974ca4-e83b-4f14-8472-83f97e9e3970 service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:27.825172 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:27.825810 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:27.826122 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:27.826404 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] PATH_INFO: `/v3/users/e2d108f5b3e64655a62cc6d25255ff87` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:27.827585 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d108f5b3e64655a62cc6d25255ff87)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:27.829559 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:28.057786 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d108f5b3e64655a62cc6d25255ff87'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.058503 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d108f5b3e64655a62cc6d25255ff87'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.065468 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d108f5b3e64655a62cc6d25255ff87'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.077456 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d108f5b3e64655a62cc6d25255ff87'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.139904 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d108f5b3e64655a62cc6d25255ff87'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.147025 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1463/2903] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:48:27 2022] DELETE /identity/v3/users/e2d108f5b3e64655a62cc6d25255ff87 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:48:28.415734 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:28.427072 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00463-1e6c-49cd-bdad-2d04c0bf7d6d service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:28.488222 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:28.488969 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:28.493478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:28.493478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] PATH_INFO: `/v3/projects/f06fcba3614a4f428778ccfa4d2ac3eb` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:28.493478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f06fcba3614a4f428778ccfa4d2ac3eb)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:28.493478 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:28.591937 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f06fcba3614a4f428778ccfa4d2ac3eb', 'request_id': 'req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.642677 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f06fcba3614a4f428778ccfa4d2ac3eb', 'request_id': 'req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.648934 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f06fcba3614a4f428778ccfa4d2ac3eb', 'request_id': 'req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.660287 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] The token cache is being invalidate because project f06fcba3614a4f428778ccfa4d2ac3eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:48:28.660543 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.661191 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.662704 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1441/2904] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:48:28 2022] DELETE /identity/v3/projects/f06fcba3614a4f428778ccfa4d2ac3eb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:48:28.666868 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:28.668361 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:28.683323 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b03afc-7c83-47aa-986a-2cae38fc735a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:28.741439 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:28.742120 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:28.742400 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:28.742673 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] PATH_INFO: `/v3/users/89ac4c03f5514311832c4de2d9ca464f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:28.743846 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89ac4c03f5514311832c4de2d9ca464f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:28.746568 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:28.926129 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89ac4c03f5514311832c4de2d9ca464f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.926840 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89ac4c03f5514311832c4de2d9ca464f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.934359 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89ac4c03f5514311832c4de2d9ca464f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.945954 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89ac4c03f5514311832c4de2d9ca464f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.948343 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:48:28.990935 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89ac4c03f5514311832c4de2d9ca464f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:28.998452 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1464/2905] 10.222.0.128 () {58 vars in 1229 bytes} [Tue Jul 26 15:48:28 2022] DELETE /identity/v3/users/89ac4c03f5514311832c4de2d9ca464f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:48:29.267298 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:48:29.268784 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:48:29.283544 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da085d7-794a-4f2b-a9ff-e4eaf73e1796 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:48:29.292848 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:48:29.342586 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2685b786dd4d2eb4cf48fedfc3b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b15b339f34b0459888ba2e587e895d4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:48:29.343237 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:48:29.343526 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:48:29.343806 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] PATH_INFO: `/v3/projects/accee6d6b3bd4180befb4f81ec0d5ca3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:48:29.345173 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=accee6d6b3bd4180befb4f81ec0d5ca3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:48:29.347811 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:48:29.431954 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'accee6d6b3bd4180befb4f81ec0d5ca3', 'request_id': 'req-464b7011-01e0-47c7-a4cd-182303992af2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:29.471236 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'accee6d6b3bd4180befb4f81ec0d5ca3', 'request_id': 'req-464b7011-01e0-47c7-a4cd-182303992af2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:29.477274 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'accee6d6b3bd4180befb4f81ec0d5ca3', 'request_id': 'req-464b7011-01e0-47c7-a4cd-182303992af2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:29.489802 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] The token cache is being invalidate because project accee6d6b3bd4180befb4f81ec0d5ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:48:29.490142 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:29.490837 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-464b7011-01e0-47c7-a4cd-182303992af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:48:29.492369 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1442/2906] 10.222.0.128 () {58 vars in 1238 bytes} [Tue Jul 26 15:48:29 2022] DELETE /identity/v3/projects/accee6d6b3bd4180befb4f81ec0d5ca3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:48:39.053252 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:48:39.486469 dv-cinder-851032-1-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:53:27.447001 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:53:27.448582 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:53:27.592328 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f856-899c-4096-86d3-1de3464f0d4f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:53:27.652734 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1def01-c62c-4f5c-9d38-715eaf7001e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e8966ae76242898bc996cd039dbe31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95e80fcc351a4e52bbcd7ccdb61baa07', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:53:27.653357 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1def01-c62c-4f5c-9d38-715eaf7001e1 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:53:27.653639 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1def01-c62c-4f5c-9d38-715eaf7001e1 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:53:27.653914 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1def01-c62c-4f5c-9d38-715eaf7001e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:53:27.655661 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4a1def01-c62c-4f5c-9d38-715eaf7001e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:53:27.667107 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1def01-c62c-4f5c-9d38-715eaf7001e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:53:27.668705 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1def01-c62c-4f5c-9d38-715eaf7001e1 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:53:27.679163 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1465/2907] 10.222.0.128 () {60 vars in 1370 bytes} [Tue Jul 26 15:53:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:53:37.674442 dv-cinder-851032-1-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}